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collection system of the National Center for Education Statistics, which has 
the legislative mission of collecting and publishing data on the condition of 
education in the United States. The NHES provides information on educational 
issues that are best addressed by contacting households rather than 
educational institutions . It is a telephone survey of the 

noninstitutionalized civilian population of the United States. Households are 
selected through random digit dialing methods, and data are collected using 
computer-assisted telephone interviewing. Approximately 60,000 households are 
screened for each administration, and those who meet certain criteria are 
sampled for more detailed interviews. This report examines the reliability of 
two components of the NHES for 1993, the School Safety & Discipline and 
School Readiness surveys. Measurement errors were estimated by reinterviewing 
a sample of respondents and asking them a subset of items from the original 
interview. Of the 2,108 cases selected, 1,879 reinterviews were actually 
conducted, for an unweighted completion rate of 89.1%. Indexes of consistency 
were prepared for specific items. Many School Safety & Discipline items had 
moderate to high inconsistencies. An appendix presents the reinterview 
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Background 

The National Household Education Survey 
(NHES) is a data collection system of the 
National Center for Education Statistics (NCES), 
which has as its legislative mission the collection 
and publication of data on the condition of 
education in the Nation. The NHES is 
specifically designed to support this mission by 
providing information on those educational 
issues that are best addressed by contacting 
households rather than schools or other 
educational institutions. The NHES provides 
descriptive data on educational activities of the 
U.S. population and offers policymakers, 
researchers, and educators a variety of statistics 
on the condition of education in the United 
States. 

The NHES is a telephone survey of the 
noninstitutionalized civilian population of the 
U.S. Households are selected for the survey 
using random digit dialing (RDD) methods, and 
data are collected using computer-assisted 
telephone interviewing (CATI) procedures. 
Approximately 60,000 households are screened 
for each administration, and individuals within 
households who meet predetermined criteria are 
sampled for more detailed or extended 
interviews. The data are weighted to permit 
estimates of the entire population. The NHES 
survey for a given year typically consists of a 
Screener, which collects household composition 
and demographic data, and extended interviews 
on two substantive components addressing 
education-related topics. In order to assess data 
item reliability and inform future NHES surveys, 
each administration also includes a subsample of 
respondents for a reinterview. 

The primary purpose of the NHES is to conduct 
repeated measurements of the same phenomena 
at different points in time, although one-time 
surveys on topics of interest to the Department of 
Education are also conducted. Throughout its 
history, the NHES has collected data in ways that 
permit estimates to be tracked across time. This 
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includes repeating topical components on a 
rotating basis in order to provide comparative 
data across survey years.. In addition, each 
administration of the NHES has benefited from 
experiences with previous cycles, resulting in 
enhancements to the survey procedures and 
content. Thus, while the survey affords the 
opportunity for tracking phenomena across time, 
it is also dynamic in addressing new issues and 
including conceptual and methodological 
refinements. 

A new design feature of the NHES program 
implemented in the NHES:96 was the collection 
of demographic and educational information on 
members of all screened households, rather than 
just those households potentially eligible for a 
topical component. In addition, this expanded 
screening feature included a brief set of 
questions on an issue of interest to education 
program administrators or policymakers. The 
total Screener sample size was sufficient to 
produce state estimates of household 
characteristics for the NHES:96. 

Full-scale implementations of the NHES have 
been conducted in 1991, 1993, 1995, and 1996. 
Topics addressed by the NHES:91 were early 
childhood education and adult education. The 
NHES:93 collected information about school 
safety and discipline and school readiness. The 
1991 components were repeated for the 
NHES:95, addressing early childhood program 
participation and adult education. Both 
components underwent substantial redesign to 
incorporate new issues and develop new 
measurement approaches. In the NHES:96, the 
topical components were parent/family 
involvement in education and civic involvement. 
The NHES:96 expanded screening feature 
included a set of questions on public library use. 

In addition to its topical components, the NHES 
system has also included a number of 
methodological investigations. These have 
resulted in technical reports and working papers 
covering diverse topics such as telephone 
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undercoverage bias, proxy reporting, and 
sampling methods. This series of technical 
reports and working papers provides valuable 
information on ways of improving the NHES. 
More information on the NHES:93 is available in 
Brick, et al. (1994b, 1994c). 

Overview 

This report examines the reliability of the 
responses to the interviews in both the School 
Safety & Discipline (SS&D) and School 
Readiness (SR) components of the NHES:93. 
Estimates from these components, like the 
estimates from every sample survey, are subject 
to both sampling error and nonsampling error. 
Sampling errors, the differences between the 
population values and the sample estimates that 
arise because data are obtained from only a 
sample of the population, are generally well 
understood and can be estimated from the survey 
data themselves. Nonsampling errors, on the 
other hand, arise from a variety of sources and 
are more difficult to measure. Important 
components of nonsampling error for the 
NHES:93 include coverage, nonresponse, and 
measurement errors. 

In this analysis, measurement errors are estimated 
by reinterviewing a sample of respondents and 
asking them a subset of the same items included in 
the original interview. The reinterview procedure 
does not account for all the measurement errors in 
the interviewing process. For example, systematic 
errors that would be made in both the original 
interview and the reinterview are not discovered 
with this approach. Examples of systematic errors 
are respondents consistently underreporting 
income by excluding interest income, or 
respondents reporting that no students brought 
weapons to school, a possible social desirability 
bias. In contrast, the statistics produced by 
comparing the original and reinterview responses 
estimate the consistency of reporting, assuming 
both interviews were conducted under the same 
general conditions. 
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A reinterview was conducted for the early 
childhood component of the NHES:91 (Brick 
and West 1992) The approach to the reinterview 
for the NHES:93 is similar to that study, but the 
methods used expand on those in the NHES:91. 
The general review of the design and analysis of 
reinterviews presented by Forsman and Schreiner 
(1991) is useful background for understanding 
the goals and methods used in the NHES:93 
reinterview. Brick et al. (1994a) discuss the use 
of reinterview data in the broader context of 
other nonsampling errors. 

In the NHES:93 reinterview study, the same 
respondents were asked to respond to the same 
items on different occasions. They may have 
given different responses at these two times. 
Such discrepancies in responses can be grouped 
into four categories: 

• Circumstances may have changed between 
the first and second interview and both 
answers, although different, may be correct. 

• The original data item may have been 
recorded incorrectly (interviewer error) or 
reported incorrectly (respondent error). 

• The reinterview data item may have been 
recorded or reported incorrectly. 

• Both the original and reinterview responses 
may have been recorded or reported 
incorrectly. 

In the NHES:93 reinterviews, discrepancies 
between some of the original interview items and 
the reinterview items were reconciled. This 
means that when the reinterview response was 
different from the original response, the 
interviewer asked the respondent to verify which 
response was correct. This process of 
reconciling the responses was done after the 
reinterview was completed, and it was done only 
for selected items. For items that were 
reconciled, there are three different responses 
available for analysis: the original response, the 
unreconciled reinterview response, and the 
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reconciled response when the original and 
reinterview responses were not identical. 

In this report, all three of these responses are 
used. Typically, the difference between the 
original response and unreconciled reinterview 
response is used to assess the consistency of 
reporting, while the difference between the 
original and reconciled responses is used to 
assess the bias in the estimate from the original 
survey. The suitability of the reconciled data for 
estimating response bias is dependent upon the 
reconciled response being more accurate than the 
original and reinterview response (Forsman and 
Schreiner 1991). This condition is examined 
using the NHES:93 data later in this report. 

An objective in many reinterview programs is to 
provide a check on interviewers who might be 
recording entire interviews without speaking to 
the respondents (U.S. Department of Commerce 
1968). Since the NHES:93 was a CATI survey 
operated in a centralized location, there was no 
need to design reinterviews to verify that 
interviews were genuine. The CATI interviews 
were closely monitored, and it was highly 
unlikely that a telephone interviewer could 
invent whole interviews. 

The primary objectives for the NHES:93 
reinterview program were 

• To identify survey items that were not 
reliable, i.e., the two interviews did not elicit 
the same response; 

• To quantify the magnitude of the response 
variance for groups of items collected from 
the same respondent at two different times; 
and 

• To provide feedback to improve the design 
of questionnaire items for future surveys. 

A subset of the original SS&D and SR 
questionnaire items was included in the 
reinterview program for the NHES:93 to reduce 
the burden on respondents who had already 



completed one or more full interviews and to 
prevent asking some questions that were 
dependent on the specific date of the interview. 
In general, the items selected were those that 
were very important substantively, were not 
highly time dependent, and were not already 
examined in previous NHES reinterviews. The 
reinterview questionnaires are reproduced in full 
in Appendix A. 

Items were selected from specific subject areas. 
Those subject areas chosen for the reinterview 
and the associated populations for each are as 
follows: 

SS&D Reinterview Subject Areas 

School characteristics (all parents) 

School environment (all) 

School safety (all) 

School discipline policy (all) 

Tobacco, alcohol, and other drugs (all) 

Alcohol and drug education (all) 

Child characteristics (all parents) 

Family characteristics (all parents) 

Community characteristics (all) 

SR Reinterview Subject Areas 

Developmental profile (preschoolers only) 

Early childhood programs (all) 

Child adjustment to kindergarten or primary 
school (kindergarten and primary only) 
Teacher feedback on child's performance and 
behavior (kindergarten and primary only) 
Special help in school (primary only) 

Health and nutrition (all) 

Entry to kindergarten (kindergarten and primary) 
Reading and television (all) 

Some of the key features of the study design of 
the NHES:93 reinterview program are described 
below. The properties of the statistics computed 
using the reinterview responses are then 
described and these statistics are presented for 
the SS&D and SR components of the NHES:93. 
The final section of the report summarizes the 



findings and provides some recommendations for 
future work. 

Study Design 

NHES:93 Sample Design 

The NHES:93 was a RDD telephone survey 
conducted with persons in a sample of telephone 
households in the 50 States and the District of 
Columbia between January and May 1993 using 
computer-assisted interviewing. First, a 
screening interview was administered to identify 
households and eligible persons within the 
households. The study included two 
components: an SS&D interview of parents of 
students enrolled in grades 3 through 12, and an 
SR interview of the parents of children from 3 to 
7 years old or 8 to 10 years old and in second 
grade or below. For the SS&D component, the 
children in grades 3 through 5 were sampled at a 
lower sampling rate than the students in grades 6 
through 12, and the parents of the younger 
children were asked only a subset of the items 
asked of parents in the higher grades. In 
addition, youths enrolled in grades 6 through 12 
were subsampled and interviewed, provided the 
parent interview for the youth was already 
completed. 

All members of households with children 18 
years of age or younger or with children enrolled 
in 12th grade or below and 20 years old or 
younger were enumerated. The appropriate 
respondent for the parent component of the 
SS&D component or the SR interview was 
identified for each sampled child; the respondent 
was the parent or guardian who knew the most 
about the child's care and education. The 
respondent for the youth SS&D interview was 
the sampled youth. 



The NHES:93 covered the noninstitutionalized 
civilian population of persons from age 3 
through those enrolled in 12th grade and their 
parent respondents in the United States. Since 
only persons in telephone households were 
surveyed, the estimates were adjusted so that the 
totals were consistent with the total number of 
persons in both telephone and nontelephone 
households. The User’s Manuals for the SS&D 
and SR data files contain more information on 
the methods used to process the interview data 
and develop the estimates (Brick et al. 1994a, 
1994b). 

Table 1 summarizes the response and completion 
rates from the NHES:93. All of the estimated 
response rates are weighted using the 
probabilities of selection. Screening interviews 
were completed with 63,844 households, with an 
estimated response rate of 82 percent. A total of 
10,117 SS&D parent interviews about youths 
enrolled in grades 6 through 12 were completed 
for a completion rate of 90 percent and an overall 
response rate of 74 percent; corresponding rates 
for the parents of 3rd to 5th graders were 89 
percent and 74 percent, respectively. There were 
also 6,504 completed interviews for the youth 
enrolled in grades 6 through 12, for a completion 
rate of 83 percent and an overall response rate of 
68 percent. The lower response rate for the 
youth was largely due to the need to complete 
the parent interview and obtain the consent of the 
parent before interviewing the youth. A total of 
10,888 SR interviews were completed, resulting 
in a 90 percent completion rate. The overall 
response rate (the Screener response rate 
multiplied by the SR completion rate) for the SR 
interviews was 74 percent. 
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Table 1. — Weighted completion and response rates for the NHES.93, by survey component 



Component 


Number of 
interviews 
sampled 


Number of 
interviews 
completed 


Weighted 

completion 

rate a 


Weighted 

response 

rateb 


Households 


76,093 


63,844 


82.1% 


82.1% 


School Safety & Discipline interview 










parents of 3rd to 5th graders 


2,882 


2,563 


89.4 


73.4 


parents of 6th to 1 2th graders 


11,650 


10,117 


89.6 


73.6 


6th to 12th graders 


8,066 


6,504 


83.0 


68.1 


School Readiness interview 


12,905 


10,888 


89.6 


73.6 



“The completion rate is the percent of eligible respondents at the particular phase of the survey divided by the number of eligible 
sampled persons. 

“The response rate is the overall percent of completed interviews; it is the screener response rate multiplied by the completion rate for 
the extended interview. 



SOURCE: U.S. Department of Education, National Center for Education Statistics, National Household Education Survey (NHES), 
spring 1993. 



Reinterview Design 

As noted earlier, reinterviews with the 
respondents to the original interview were 
intended to supply data about the reliability of 
their responses. A random sample of parent and 
youth respondents to completed interviews was 
selected to accomplish this objective. However, 
not all interviews were eligible for a reinterview. 

A household and the interviews within the 
household were eligible for sampling for 
reinterview once all of the SS&D and SR 
interviews in the household was completed. If 
some of the interviews in the household were not 
completed even though others were, then none of 
the interviews in the household was eligible for 
reinterview sampling. This occurred most often 
when the sampled person could not be contacted 
at a convenient time to complete the interview. 
This restriction in the sample was implemented 
to prevent the reinterviewing from disrupting the 
completion of the original interviews. 



Another exclusion for the reinterview sampling 
involved refusal and language problem cases. If 
the respondent initially refused to complete the 
original interview but later completed the 
interview in refusal conversion, the interview 
was not eligible for reinterview sampling. 
Similarly, if the original interview was coded as 
a language problem case but later completed 
(perhaps by a bilingual interviewer), the case 
was excluded from reinterview sampling. This 
restriction was at the household level rather than 
at the individual interview level; i.e., if any 
interview in the household was a refusal or 
language problem case, then the household was 
not eligible for sampling. 

In all, 22 percent of households were excluded 
from the reinterview sampling for these reasons. 
Most of the exclusions (16.2% of the 
households) were because not all the interviews 
in the household had been completed. The other 
(6.0%) households were excluded because of the 
refusal and language problem restrictions. 



The only other restriction in the sample for the 
reinterviews was the exclusion of interviews for 
children with circumstances that were so 
different that they did not follow the normal 
interview patterns. In particular, parents of 62 
children who were being educated in home 
school, originally respondents in the SR 
interview, and 77 youth who were not living with 
a parent or guardian (called emancipated youth), 
originally respondents in the SS&D, were not 
eligible for the reinterview. Since there were so 
few of these interviews and the questions asked 
in these interviews were different from the other 
interviews, these interviews were excluded from 
the reinterview. All exclusions were made prior 
to sampling for the reinterview. 

Because of the restrictions in reinterview 
sampling, the results of the reinterview program 
should not be generalized to all respondents. 
However, there is value in examining reinterview 
results for those who were eligible for sampling. 

Once the household and the interviews within the 
household became eligible for reinterview, 
specific interviews were sampled. The sampling 
for reinterview was not done until at least 2 
weeks after all original interviews in the 
household were finalized, although this time 
restriction was relaxed at the end of the data 
collection period so that all eligible households 
had an opportunity to be sampled. 

To limit the burden on a household, no more than 
one case was sampled for reinterview from the 



same household, although any of the completed 
interviews in the household could have been 
sampled for the reinterview. The target sample 
size for the reinterview was set for each type of 
interview in the NHES:93 ; for parents of 3rd to 
5th graders, the goal was 250 completed 
reinterviews; for parents of 6th to 12th graders, 
the goal was 250 completed reinterviews; for 6th 
to 12th graders, 500 completed reinterviews; and 
for the SR, the goal was 1,000 completed 
reinterviews. These target sample sizes were set 
after reviewing the results of the NHES:91 
reinterview (Brick and West 1992) and 
determining that sample sizes of more than 500 
completed reinterviews were needed to assess 
more detailed characteristics. Thus, sample sizes 
of 1,000 completed reinterviews were assigned 
to both the SS&D and SR components in order to 
allow for the fact that subgroups were of interest 
in the analysis and that some items of interest 
were subgroup-specific, especially in the SR 
interview. The interviews were sampled within 
household at the same rate for each type of 
interview to achieve the desired sampling rate. 

A sample of 2,108 cases was selected for 
reinterview, and 1,879 cases were completed for 
an unweighted completion rate of 89.1 percent. 
Table 2 shows the number of cases sampled for 
reinterviews, the number of those that resulted in 
completed reinterviews, and the unweighted 
completion rate by the type of interview. The 
completion rates are uniformly high, ranging 
from 88 percent to 90 percent. 
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Table 2.— Reinterview sample sizes, completed interviews, and unweighted completion rates, by major 
path 



Major path 


Completed 

original 

interviews 


Reinterview 
sample size 


Completed 

reinterviews 


Unweighted 
reinterview 
completion rate 


All 


30,072 


2,108 


1,879 


89.1% 


SS&D 

parents of 3rd to 5th graders 


2,563 


256 


227 


88.7 


parents of 6th to 1 2th graders 


10,117 


315 


277 


87.9 


6th to 12th graders 


6,504 


560 


493 


88.0 


SR 


10,888 


977 


882 


90.3 



SOURCE U.S. Department of Education, National Center for Education Statistics, National Household Education Survey (NHES), 
spring 1993. 



The main reason for not completing the 
reinterview was the inability of interviewers to 
contact the respondent during the reinterview 
time period. This includes never making contact 
with the respondent, reaching a disconnected 
phone or phone number that had been changed, 
and attempting to reach someone who had 
moved to a new household with no telephone or 
no forwarding number. Only about one-third 
(one-quarter for youth respondents) of the 
nonresponse was due to refusals to participate in 
the reinterview. If the respondent refused, no 
further efforts were made to complete the 
reinterview. 

As noted above, the reinterview was originally 
designed to be conducted at least 2 weeks after 
the completion of the original interview. Table 3 
shows the actual number of days between the 
original interview and the reinterview for each 
type of respondent. About 40 percent of the 
reinterviews were conducted between 2 and 4 
weeks from the date the original interview was 
completed. It took longer than 4 weeks to 
complete many reinterviews because all 
extended interviews in a household had to be 
completed before sampling for the reinterview. 
In addition, about 9 percent of the reinterviews 



were completed less than 15 days after the 
original interview because the time between the 
reinterview and the original interview was 
shortened at the end of the data collection period. 
Bailar (1968) discusses how the time between 
the interview and reinterview might influence the 
estimates, but no analysis of this effect has been 
done with these data because of the relatively 
small number of reinterviews completed for each 
type of respondent. 

The reinterview was conducted using the same 
CATI system used in the original interview, 
modified to display the reinterview items. The 
interviewers read identical items to the same 
parent/guardian or youth who completed the 
original interview. No substitutions were 
allowed for the original respondent. For the SR 
component, 83 percent of the respondents were 
the child's mother, 14 percent were the child's 
father, and the remaining 3 percent were other 
persons such as grandparents, stepparents, or 
guardians. For the SS&D parent interviews, 80 
percent of the respondents were the youth's 
mother, 16 percent were the youth's father, and 4 
percent were other persons. The youth sampled 
as the subject of the SS&D percent interview was 
the respondent for the youth interview. 
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Table 3. — Number of days between completion of original interview and reinterview 









SS&D 


SS&D 


SS&D 












Parents of 3rd to 


Parents of 6th to 


6th through 12th 








Total 


5th graders 


12th graders 


graders 


SR 


Number of days 


Number 


Pet. 


Number 


Pet. 


Number 


Pet. 


Number 


Pet. 


Number 


Pet. 


Total 


1,879 


100 


227 


100 


277 


100 


493 


100 


882 


100 


Less than 8 


30 


2 


2 


l 


0 


0 


15 


3 


13 


1 


8 to 14 


131 


7 


11 


5 


23 


8 


47 


10 


50 




15 to 21 


234 


12 


36 


11 


26 


9 


67 


14 


105 


12 


22 to 28 


519 


28 


61 


27 


72 


26 


163 


33 


223 


25 


29 to 35 


297 


16 


28 


16 


48 


17 


64 


13 


157 


18 


36 to 42 


363 


19 


52 


23 


52 


19 


78 


16 


181 


21 


43 to 49 


160 


9 


20 


9 


23 


8 


34 


7 


83 


9 


50 or more 


145 


8 


17 


7 


33 


12 


25 


5 


70 


8 



NOTE: Percentages may not sum to 100 because of rounding. 



SOURCE: U.S. Department of Education, National Center for Education Statistics, National Household Education Survey (NHES), 
spring 1993. 



After all of the items for the reinterview were 
asked, the original and reinterview responses 
were compared automatically by the computer 
for a subset of items. We refer to this process as 
reconciliation. Reconciliation was conducted for 
items in the SS&D reinterviews that concerned 
school environment, drug/alcohol education and 
use, and incidents of victimization at school. In 
the SR reinterviews, reconciliation was 
conducted for a subset of items — those 
associated with early childhood program 
participation, children receiving help in school, 
and events in the previous week. Items were 
selected because they were new to the NHES and 



no previous information on their response 
variability was available or because they were 
key indicators for young children, e.g., reading 
or preschool participation. 

Up to the point of reconciliation, the interviewer 
was unaware of the responses given by the 
respondent in the original interview. For any of 
the selected items that had different responses in 
the original and reinterview surveys, a 
reconciliation screen appeared and the 

interviewer asked the respondent which answer 
was best. A typical screen used to resolve the 
differences is shown in Exhibit 1 . 



Exhibit 1. — Typical CATI reconciliation screen 



60.095 P56CK 

We asked you if {child's} school has a written discipline policy. One time we recorded YES 
and one time we recorded NO. What is the best answer? 

( ) 

1. YES 

2. NO 

3 

ERIC 
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Analysis Methods 

Several statistics have been developed to 
examine various aspects of the reliability of 
reporting using original reinterview responses. 
Three of these statistics used for most of the 
analysis of the responses from the NHES:93 in 
this report are the gross difference rate, the net 
difference rate, and the index of inconsistency. 
In addition, three other statistics used in the 
report are the correlation coefficient, the percent 
agreement, and the agreement ratio. All six 
statistics are described below. 

The gross difference rate for a binary variable is 
the percentage of items with different responses 
in the two interviews. Thus, it is an estimate of 
the reliability or consistency of reporting. The 
net difference rate takes account of offsetting 
misclassifications. When the reinterview is error 
free, then the net difference rate estimates the 
bias in the estimate. The index of inconsistency 
is a less familiar statistic that is a relative 
measure of response variability. In 
circumstances described later, the index can be 
used to measure the proportion of the total 
variability that arises due to random response 
error (U.S. Department of Commerce 1968; 
Forsman and Schreiner 1991). 

These statistics are typically computed based on 
the number of sample cases reported as having a 
particular characteristic in the original survey 
and in the reinterview. This approach is valid for 
simple random sampling or when the goal of the 
analysis is to evaluate and quantify response 
variability of the population of survey 
respondents. When the goal is to provide 
estimates of response variability of the national 
estimates, it is more appropriate to estimate these 
statistics using weights that adjust for the 
probability of selection. Since this was a main 
objective of the NHES:93 reinterview, the 
weight for each completed interview developed 
for the analysis of the original interview was 
used for all the analysis in this report. No 
additional adjustments to the weights were made 



to account for sampling or nonresponse in the 
reinterview. The sampling errors of all the 
estimates were computed using a replication 
method. The full sample and replicate weights 
are described in the NHES:93 User’s Manuals 
(Brick etal. 1994b, 1994c). 

Table 4 shows the general format of the possible 
reporting outcomes from the original and 
reinterviews when the item has only two possible 
values. From tables formatted in this fashion, it 
is possible to estimate several characteristics 
relevant to the consistency of the reporting 
between the original survey and the reinterview. 
For example, the off-diagonal cells estimate the 
proportion of responses that were reported 
differently in the original interview and the 
reinterview. Since the statistics computed in this 
report are based on weighted data, the values in 
the cells are actually weighted sums of the 
number of cases rather than the raw number of 
cases. The definitions of the statistics used in 
this report are given below, where the cell counts 
are the estimated totals. 

Gross Difference Rate. The gross difference is 
equal to the weighted percentage of cases 
reported differently in the original interview and 
the reinterview. The gross difference rate is the 
ratio of the gross difference divided by the 
estimated total number of cases. The gross 
difference rate can be written algebraically as 

i *(*-*>? (l) 

£ 'ty- 

where JC], is the response to the original 
interview for case i, * 2 ,• is the response to the 
reinterview for case i, and w f is the original 
interview weight for case i, described above. 
Thus, the gross difference rate is the average 
squared difference between the responses. 

* 

If either the original response or the reinterview response 
was missing for an individual item, the statistics were 
computed excluding the case. Imputed values were not 
used in this report. Since the item response rate was very 
high for both the interview and the reinterview, the 
exclusion of the missing values was not a significant 
problem. 
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For characteristics that have exactly two possible 
outcomes, the gross difference rate can be 
written as a percentage using the terms from 
Table 4 as 

G% = 100 — . (2) 

n 

This can easily be seen to be a special case of (1) 
where the x t terms only take on the values of 0 or 
1. For binary data, it is clear from (2) that the 
gross difference rate is an estimate of the 
percentage of cases not reported the same in both 
interviews, i.e., those falling in the off-diagonal 
cells. 

For items with more than two nominal response 
categories, several options are available. One is 
to simply make the responses binary by 
collapsing the response categories to form a 
dichotomy. For example, in this report, response 



categories concerning whether the child was 
challenged at school were collapsed from 5 
response categories to two. 

Another alternative is to create sets of new 
binary variables corresponding to each of the 
response categories of the original item. For 
example, if there are three response categories 
(a, b, and c), then new variables could be created 
such that the first new variable has the value ‘ 1 ’ 
if the response to the original variable is category 
‘a’ and it is ‘O’ otherwise; the second new 
variable has the value ‘1* if and only if the 
response to the original variable is category ‘6.’ 
The first option was used in this report because 
creating multiple variables often resulted in 
having a small percentage of the sampled cases 
in one of the categories, which, as discussed 
below, adds complexity to the analysis. The 
response categories that were collapsed for the 
reinterview analysis are noted on the reinterview 
questionnaires in the appendix. 



Table 4. — General format of interview-reinterview results 





Original interview 




Reinterview 


Number of 
cases with 
characteristic 


Number of 
cases without 
characteristic 


Total 


Number of cases 
with a characteristic 


a 


b 


a + b 


Number of cases 
without a characteristic 


c 


d 


c + d 


Total 


a + c 


b + d 


n=a+b+c+d 



SOURCE: U.S. Department of Education, National Center for Education Statistics, National Household Education Survey (NHES), 
spring 1993. 



The gross difference rate divided by 2 is an estimate 
of the response variance for an item where response 
variance is defined as the variation associated with 
the responses to the same item when the survey is 



repeated under the same general conditions. In 
essence, this means that the interview and 
reinterview responses are independent of each 
other and the other factors that affect the responses 
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( 3 ) 



are the same for both interviews so that the 
responses have the same distribution. Forsman and 
Schreiner (1991) discuss these assumptions in more 
detail. 

To aid in the presentation of the gross difference 
rates, the following general rule is used to 
categorize the response variance as measured by the 
gross difference rate: 

• A gross difference rate of less than 10 percent is 
low response variance; 

• A gross difference rate between 10 and 20 
percent is moderate response variance; and 

• A gross difference rate above 20 percent is high 
response variance. 

This rule was developed! using the gross difference 
for a binary variable as a guide. Thus, a low gross 
difference rate means that fewer than 1 0 percent of 
the respondents are misclassified; a moderate rate 
means between 10 and 20 percent are misclassified; 
and a high rate means more than 20 percent of the 
respondents are classified differently in the original 
and reinterview. 

Since the gross difference rate is an absolute 
measure of the measurement error, this rule does not 
account for the fact that 5 percent misclassified is a 
much more serious response problem for a 1 percent 
statistic than a 50 percent statistic. To account for 
the relative size of the estimate, the rule stated 
above is only applied to estimates between 20 and 
80 percent. Outside of this range, the gross 
difference rate and other measures of data quality 
should be considered with respect to the size of the 
estimate. 

Net Difference Rate. As with the gross difference 
rate, the net difference rate can be computed for 
both continuous variables and those that have only 
two values. The net difference rate for a continuous 
variable is 



»/_ ZWi(x v -x 9i ) 

H w i 

where the variables are defined as in (1). The net 
difference rate is thus the average difference 
between the original and reinterview. 

For the binary case, the net difference is the 
difference between the weighted count of cases with 
a characteristic as reported in the original interview 
and the weighted count of cases in the reinterview. 
That is, (a + c) - {a + b) = c-b, using the terms in 
Table 4. While the gross difference includes 
differences in both directions, the net difference is 
the nonoffsetting part of the gross difference. The 
net difference rate, expressed as a percentage, is 

N% = 100 S^-L . (4) 

n 

If the reinterview response is the “true” value, then 
the net difference rate is an estimate of the response 
bias of the estimate. Generally speaking, this was 
not the case in the NHES:93 since the reinterview 
responses were collected under the same general 
conditions as the original interview. The 
reinterviews were conducted by regular NHES:93 
interviewers, that is, those who had conducted the 
original interviews. No control was implemented to 
prevent an interviewer from conducting both the 
original interview and the reinterview with the same 
respondent. However, given the number of 
interviews conducted in the NHES:93 and the size 
of the interviewing staff (about 300), it is unlikely 
that this occurred often, if at all. No changes were 
made in the interview itself other than eliminating 
some of the items, and the respondents were the 
same for both interviews. As a result, there is no 
reason to assume that the reinterview responses are 
closer to the “true” values than are the original 
interview responses. 
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Even though the net difference rate computed using 
the original and unreconciled responses is generally 
not a valid estimate of response bias, it can be used 
to evaluate one of the assumptions under which the 
gross difference rate is a valid estimate of response 
variance. As noted above, if the reinterview is an 
independent replication of the original interview, 
then the gross difference rate divided by 2 is an 
unbiased estimate of the simple response variance. 
The net difference rate computed from the 
unreconciled reinterview data can be used to 
examine whether the two interviews result in similar 
distributions. If the two interviews are independent, 
then the expected value of the net difference rate 
should be equal to 0. Biemer and Forsman (1992) 
give more details on this. The net difference rate 
computed from the original and unreconciled 
responses is used later to evaluate this assumption. 



Index of Inconsistency. The index of inconsistency 
is the ratio of the variance of response errors to the 
total variance of the statistic. For continuous 
variables, the index of inconsistency is 



1 = 



G 

„2 , „2 

•*1 + i 2 



(5) 



where G is the gross difference rate defined above 
for continuous variables, sf is the sample variance 



9 

for the original interview, and sf is the sample 
variance for the reinterview. For binary data, the 
index can be expressed as a percentage as 

P/a = 100 — — — (6) 

2np(\-p) 



where p = 



a + c 

n 



The net difference rate computed from the original 
and the reconciled responses is sometimes used to 
estimate response bias. Reconciliation is an explicit 
effort to discover a more accurate response to the 
interview question. Typically, the interviewers 
doing the reconciliation are supervisors or more 
highly trained individuals who are expected to 
obtain more precise responses. Brick and West 
(1992) found that there was no evidence that 
reconciled responses were more accurate than the 
original responses in circumstances very similar to 
those in the NHES:93. Part of the reason for that 
finding may be related to the use of the same 
interviewer pool for conducting the interview and 
the reinterview. Forsman and Schreiner (1991) 
discuss the conditions under which reconciled 
reinterviews may be treated as providing valid 
estimates of response bias. Despite concerns about 
the assumption that the reconciled response is more 
accurate than the other responses, the net difference 
rates computed from the original and the reconciled 
reinterview values are the only measures of 
response bias available from this study and are 
presented later with the other estimates from the 
study. 



The index was originally developed as a way of 
evaluating the proportion of total variability 
contributed by random response error. As noted 
earlier, the gross difference rate (1) divided by 2 is 
an unbiased estimate of the simple response 
variance if the observations from the two interviews 
are independent and identically distributed (Biemer 
and Stokes 1991). The index of inconsistency is 
therefore a ratio of the simple response variance and 
total variance of the estimate. In (6), the quantity 
p(\-p) is an estimate of the total variance for a 
binary variable. 

It should be recognized that the index of 
inconsistency (6) can take on values greater than 
100. This can happen because the response variance 
may actually reduce the overall variability in the 
estimate. Hansen, Hurwitz, and Pritzker (1964) 
showed precisely this phenomenon for a binary 
random variable. 
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It is possible, and even likely, that the responses to 
the reinterview may be affected in some ways by the 
original interview experience. This conditioning of 
respondents means that the assumption of 
independent and identically distributed responses to 
the interviews may not be fully satisfied. 
Nevertheless, the index is a valuable measure of the 
relationship between response error and sampling 
error. For this reason, the index of inconsistency is 
used in the analysis of the NHES:93 reinterview 
data. 

The index of inconsistency is a relative measure 
since the gross difference rate (an absolute measure) 
is divided by a term that depends on the variance of 
the estimate. Note that as the estimated percentage 
in the category (p ) becomes extreme (close to 0 or 
1), then the denominator of the index becomes very 
small. As a result, even a small gross difference 
rate can result in a very large index. For example, 
assuming a constant value for the gross difference 
rate, the value of / differs by a factor of about 25 
when the value of p varies from 1 percent to 50 
percent. Thus, the index is most useful for estimates 
of characteristics between 20 and 80 percent 
because in this range the quantity p(\-p) is relatively 
constant, varying only between 16 and 25 percent. 

For estimates of characteristics that are between 20 
and 80 percent, a useful rule for interpreting the 
index is given below: 

• An index of less than 20 is low relative response 
variance; 

• An index between 20 and 45 is moderate 
relative response variance; and 

• An index above 45 is high relative response 
variance. 

This rule was used previously in the NHES:91 
(Brick and West 1992) and a similar classification 
was used by Bushery, Royce, and Kasprzyk (1992). 
Bushery, Royce, and Kasprzyk classified an index 







between 20 and 50 percent as moderate. The 
classification scheme given above is used for all the 
items for both the SS&D and the SR components of 
the NHES:93. The index is not used to compare 
reliability across items for estimated percentages 
outside the 20 to 80 percent range. 

Other Statistics. Two other statistics that are 
sometimes used to assess the reliability of responses 
with reinterview data are the correlation coefficient 
and the percent agreement. These statistics are 
presented with the tables for those analysts who are 
familiar with these measures. The agreement ratio 
is the third additional statistic used in this report. 

The Pearson correlation coefficient is the ordinary 
correlation computed for continuous variables. The 
estimated correlation between the original and 
reinterview responses is 

£(*!« -*l)(*2f- *2) (?) 

^Z(*l<-*l) 2 Z(*2< ~*2) 2 

where the variables are defined as in (1). Note that 
the correlation coefficient is unweighted. While a 
weighted version of the statistic could be used, the 
unweighted correlation has been used in most 
published reports when this statistic is used. The 
correlation coefficient ranges between positive and 
negative 1. A value near positive or negative 1 
shows a strong linear relationship between the 
variables and a value of 0 shows no linear 
relationship. In reinterview studies, a high positive 
linear relationship between the original and 
reinterview is indicative of consistent reporting of 
the item. 

Another measurement statistic that is sometimes 
used for binary variables is the percent agreement. 
The percent agreement for binary variables is 

PA = lOO^i^- (8) 

n 
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where the values of a, d, and n are taken from Table 
4. This statistic is the weighted percent of all 
observations that are reported the same in both the 
interview and reinterview. This statistic is 
sometimes called the index of crude agreement 
(Forsman and Schreiner 1991) and is the 
complement of the gross difference rate for binary 
variables. 

The last statistic used in this report is called the 
agreement ratio. It is the ratio of the percent 
agreement computed using unreconciled and 
reconciled reinterview responses to the percent 
agreement computed using original and reconciled 
data. This statistic can be written as 



where PA\ is the percent agreement of the 
unreconciled and reconciled responses and PA 2 is 
the percent agreement of the original and reconciled 
responses. 



Index of Inconsistency and Correlation 

The statistics used to estimate the reliability of the 
data from the NHES:93 are based on what 
Biemer and Stokes (1991) call the survey research 
terminology. Some of these statistics may be less 
familiar to analysts coming from other backgrounds. 
The authors discuss the relationships between these 
statistics and the terms used in different disciplines 
for similar concepts. 

The index of inconsistency is perhaps the most 
difficult statistic to appreciate for those not familiar 
with the survey research terminology. Fortunately, 
the index is related to a much better known statistic, 
the Pearson correlation coefficient. Since most of 
the variables in the NHES:93 are binary, the 
relationship between the two statistics is examined 
below for binary variables. 

For a binary variable, the correlation coefficient 
given in (7) can be written using the terms from 
Table 4 as 



For example, suppose that 90 percent of the 
responses to an item were the same for the original 
and reinterview. Of the 10 percent with differences, 
suppose that in 8 percent of the cases the reconciled 
response was the same as the unreconciled response 
while in the remaining 2 percent the reconciled 
response was the same as the original interview. 
The agreement ratio for this example would be 4 
(8% 2%). The agreement ratio indicates whether 

the reconciled values were more consistent with the 
original or the unreconciled reinterview response. 
An agreement ratio greater than 1.0 means that a 
greater percent of the reconciled responses agreed 
with the unreconciled reinterview response than 
with the original interview. 



P = 



a - 



(a+c)(a+A) 



a - npp' 



( 10 ) 






, a+c , , a+b 

where p = and p = 

n n 



The similarity between the index (6) and the 
correlation expressed in this fashion is now more 
apparent. The denominators of both (6) and (10) are 
variance terms. If the proportion of cases with the 
characteristic are approximately the same in the 
original interviews (p ) and the reinterview (/>'), 
then the denominators are equal. 
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In fact, if the reinterview is a replication of the 
original interview (e.g., the samples are independent 
and have the same distribution) then the relationship 
is very simple and direct. Under these conditions, 
p = p' and the index of inconsistency equals one 
minus the correlation coefficient, if simple random 
sampling is used. 

The numerators of the two statistics are closely 
related, but not identical. To show the relationship 
more clearly, one can express the numerator of the 
index (6) as 

b + c 



= ( 11 ) 



The numerators of both the index and the correlation 
coefficient are the a term from Table 4 minus a term 
involving the proportions of cases having the 
characteristic in the interview and reinterview. The 
terms being subtracted are different for the two 
statistics; for the index the term subtracted is of the 
same magnitude as p, while for the correlation the 

term subtracted is the magnitude of p . 
Nevertheless, the two statistics are often highly 
correlated. The correlation is negative because of 
the negative term in (1 1). 



The relationship can be seen by examining some 
graphs of the statistics computed from the 
NHES:93. Figures 1 through 4 are plots of the 
index of inconsistency and the correlation for the SR 
and the three different categories of respondents 
from the SS&D. In all cases, the relationship is 
nearly linear and negative. This relationship holds 
even in the tails of the distribution below 20 percent 
and above 80 percent. The correlation was 
computed from unweighted data while the index 
was computed from weighted data, since these are 
the ways the statistics are commonly reported in the 
analysis of reinterview data. These figures show 
that users familiar with the correlation can, for 
practical purposes, consider the index of 
inconsistency a different parameterization of the 
correlation coefficient for the NHES:93 data. To 
aid these users, correlations are reported in the 
tables of findings. 

In contrast, the gross difference rate is an absolute 
measure of response variance and is not expected to 
have a linear relationship with the correlation. 
Figure 5 is a scattergram of the gross difference rate 
and the correlation for the items in the SR 
reinterview. As expected, the relationship is not 
linear. Scattergrams for the SS&D respondents (not 
shown) are similar. 



O 

ERIC 
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Figure 1.— Scattergram of the index of inconsistency and the 
correlation for the SR reinterview 




SOURCE: U.S. Department of Education, National Center for Education Statistics, National Household Education Survey (NHES), 
spring 1993. 



Figure 2.— Scattergram of the index of inconsistency and the 
correlation for parents of 6th to 12th graders in the SS&D reinterview 




SOURCE: U.S. Department of Education, National Center for Education Statistics, National Household Education Survey (NHES), 
spring 1993. 
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Figure 3.~Scattergram of the index of inconsistency and the correlation 
for parents of 3rd to 5th graders in the SS&D reinterview 




SOURCE: U.S. Department of Education, National Center for Education Statistics, National Household Education Survey (NHES), 
spring 1993. 



Figure 4.-- Scattergram of the index of inconsistency and the 
correlation for 6th to 12th graders in the SS&D reinterview 




SOURCE: U.S. Department of Education, National Center for Education Statistics, National Household Education Survey (NHES), 
spring 1993. 
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Correlation 



Figure 5.— Scattergram of the gross difference rate and the correlation 

for the SR reinterview 




SOURCE: U.S. Department of Education, National Center for Education Statistics, National Household Education Survey (NHES), 
spring 1993. 
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Measurement Errors in the NHES:93 

The statistics described above are now used to 
evaluate measurement errors for the SS&D and SR 
components of the NHES:93. For both components, 
only items that were answered by 10 or more 
respondents in both the original and reinterview are 
included. Since the primary goal of the reinterview 
was to assess response variance, the gross difference 
rates and indexes of inconsistency are given first, 
followed by net difference rates as estimates of 
response bias. In the analysis of the reliability of 
the SR items, the reliability for two composite 
variables from the SR are examined and compared 
to the reliability of the separate items. Some 
methodological issues associated with the 
reconciliation process and the validity of the 
measures of response variance and bias from the 
reinterview are then examined. 

Response Variability in the SS&D 
Component 

Three statistics measuring response variability, the 
gross difference rate, the index of inconsistency, and 
the Pearson correlation coefficient, were computed 
for each item in the SS&D survey. Because many 
items were included in the reinterview, four 
groupings of items are used to present these 
statistics in a more meaningful manner. The 
groupings are items about the general environment 
of the school, drug and alcohol education, drug and 
alcohol use (these were not asked for parents of 3rd 
to 5th graders), and incidents related to safety and 
discipline. 

Table 5 uses these groupings to summarizes the 
response variability statistics computed for the 
SS&D. Detailed tables that present the statistics for 
each item follow: Table 6 shows the statistics for 
parents of 3rd to 5th graders; table 7 shows the 
statistics for parents of 6th to 12th graders; and table 
8 shows the estimates for 6th to 12th graders. 
Standard errors for the gross difference rate and 
index of inconsistency are included in the detailed 
tables along with the estimated percentage of 
respondents in the first response category for each 
O 



item in the original interview. The wording of items 
in the SS&D interviews were not always identical 
for parents and youth (see the questionnaire in the 
appendix). For items concerning incidents at 

school, parents were asked “have you heard of ...” 
and youth were asked “do you know of...” 
Wording for items concerning the presence of drugs 
and alcohol also varied in this way. Similarly, for 
the question on safety measures at school, parents 
were asked, “do you know if (child’s) school...,” 
whereas youth were asked, “does your school...” 
These variations in wording were developed to 
recognize distinctions between first-hand and 
second-hand knowledge by parents and students. 

As shown in Table 5, the median gross difference 
rates and the indexes of inconsistency are smaller 
for the general environment items than for the other 
groupings of items. The median gross difference 
rates for the general environment items are 10 
percent or less, while the median rates for the other 
groupings are in the range of 12 to 25 percent. 

The median values of the response variance 
statistics for each grouping are of roughly the same 
magnitude for all three types of respondents. In 
particular, the median gross difference rates and 
indexes of inconsistency for the parents of 6th 
through 12th graders and the 6th through 12th 
graders themselves are comparable. This is 
important because there was concern that 6th 
through 12th graders might not be very reliable 
respondents. The consistency between their 
measurement error statistics and those for their 
parents shows that neither type of respondent was 
more reliable than the other for these items. 

Several hypotheses could explain the relatively high 
response variability for the items about incidents. 
One possibility is that respondents may not have 
been sure about the specific intent of the questions 
and their answers may have reflected this ambiguity. 
Another possibility is that the high variability for 
these items may be related to the task the respondent 
was asked to perform. The respondent was required 
to remember if a specific incident occurred and, if 
so, whether the event happened within the past 12 
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months. This may be a more difficult 
cognitive task than is required for other items 
that only ask whether or not an event happened. Yet 
another possibility is that some of these events are 



related to the time of the interview and the 
reinterview may be collecting information on 
incidents that occurred after the original interview, 
for example, a high-profile crime at a school. 



Table 5. — Summary of gross difference rates, indexes of inconsistency, and correlations based on 
original and unreconciled reinterview responses from the SS&D, by groupings of items 



Type of respondent and item 


Number 

of 

estimates 


Median gross 
difference rate 


Median 
index of 
inconsistency 


Median 

correlation 


Parents of 3rd through 5th graders 










General environment items 


13 


5.8 


32.4 


.68 


Drug and alcohol education items 


5 


24.1 


50.4 


.46 


Incidents 


10 


22.0 


56.2 


.49 


Parents of 6th through 12th graders 










General environment items 


16 


8.0 


30.4 


.68 


Drug and alcohol education items 


5 


22.2 


46.5 


.51 


Drug and alcohol use items 


3 


11.6 


47.1 


.52 


Incidents 


15 


18.9 


44.5 


.57 


6th through 12th graders 










General environment items 


8 


10.1 


35.7 


.57 


Drug and alcohol education items 


4 


25.4 


52.8 


.45 


Drug and alcohol use items 


3 


12.1 


30.6 


.69 


Incidents 


21 


16.3 


46.4 


.56 



NOTES: Gross difference rates less than 10 are considered Low, between 10 and 20 are Moderate, and greater than 20 are High. 
Indexes of inconsistency less than 20 are considered Low, between 20 and 45 are Moderate, and greater than 45 are High. The 
indexes of inconsistency for estimates outside the range of 20 and 80 percent are generally not comparable. 

SOURCE: U.S. Department of Education, National Center for Education Statistics, National Household Education Survey (NHES), 
spring 1993. 



Another hypothesis is that many of the incidents, 
drug and alcohol education, and drug and alcohol 
use items were subject to comprehension problems 
that did not affect the items in the general 
environment grouping. Since both the parents and 
youth responded with about the same level of 
consistency, the issue is the ability of respondents 
to consistently interpret the intent of the items 
rather than a more basic concern about whether or 



not the respondents understood the words used in 
the interview. For example, respondents may have 
had a vague idea of what was meant by bullying 
and not responded consistently because they 
comprehended the item differently when asked 
twice. This is more a question of respondents 
interpreting the meaning of the word “bullying” 
consistently rather than understanding it. 
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A general finding that helps summarize the 
analysis for the statistics in the detailed tables is 
that the index of inconsistency tends to be high 
when the gross difference rate is high. In a few 
cases the index is high when the gross difference 
rate is low or moderate, but this is usually because 
the estimated percentage is extreme (close to 0 or 
100 percent). As noted earlier, these extreme 
percentages correspond to the situations in which 
the gross difference rate, the index of 
inconsistency, and the correlation coefficient may 
not be as useful for comparison purposes. In the 
tables of the individual items for the SS&D, a low 
value for one of the statistics and a high value for 
the other estimate occurs only eight times. In all 
eight cases, the estimated percentage is less than 
10 percent or greater than 88 percent. 

Parents of 3rd to 5th Graders. The response 
variance statistics for items completed by parents 
of 3rd to 5th graders are shown in Table 6. The 
estimates for the general environment items are 
generally low. For the 13 general environment 
items, none of the gross difference rates is high, 
and only 2 are moderate. The indexes of 

inconsistency for these items also indicate no 
significant problems: 2 of the 13 are high, 7 are 
moderate, and 4 are low. The two indexes of 
inconsistency that are high have estimated 
percentages greater than 90 percent. 

The alcohol and drug education items have high 
estimates of response variance. Three of the five 
alcohol and drug education items have high gross 
difference rates and indexes of inconsistency and 
the remaining two items have moderate estimates. 
For the questions about incidents, 3 of the 10 items 
have low gross difference rates, 1 has a moderate 
gross difference rate, and 6 have high gross 
difference rates. The indexes of inconsistency for 
the 8 items with percentages in the 20 to 80 
percent range are high for 6 of the items and 
moderate for 2. The 6 items with both a high gross 
difference rate and index of inconsistency are 
incidents of stealing that happened to the child 
(SSTEYOU), bullying and bullying that happened 
to the child (SSBULLY, SSBULYOU), physical 
attacks or fights the child saw or happened to the 



O 




child (SSATTSEE, SSATTYOU), and interference 
with learning (SSINCDNT). 

Parents of 6th to 12th Graders. Table 7 presents 
the response variance statistics for the interviews 
with the parents of 6th to 12th graders. The 
patterns are similar to those of parents of 3rd to 5th 
graders and the general statements hold quite well 
for these respondents. Of the 16 general 
environment items, none has high gross difference 
rates and 9 have low gross difference rates. Of the 
5 items with a high index of inconsistency, none is 
for estimated percentages in the range of 20 to 80 
percent. 

All three items about drug and alcohol usage have 
moderate gross difference rates and the five items 
on alcohol and drug education have either 
moderate or high gross difference rates. The 
indexes of inconsistency for those items with an 
estimate in the 20 to 80 percent range for both 
groupings of items are either moderate or high. 

The estimates for the items in the incidents 
grouping are generally large. Only 3 of the 15 
items have low gross difference rates, 7 have 
moderate gross difference rates, and 5 have high 
gross difference rates. None of the items has a low 
index of inconsistency, while 7 of the 12 items 
with estimated percentages in the 20 to 80 percent 
range have moderate indexes, and 5 have high 
indexes. The incidents items with both a high 
gross difference rate and index of inconsistency 
are those about the child's witnessing incidents of 
robbery, bullying, and physical attacks on others 
(SSFORSEE, SSBULSEE, SSATTSEE) and the 
item about the impact the incidents had on the 
learning environment (SSINCDNT). 

Sixth to 12th Graders. Interviewing children as 
young as 1 1 years was considered a more difficult 
task than interviewing adults. Consequently, 
extensive testing of the wording of the questions 
was undertaken prior to the survey to help avoid 
problems, especially for interviews with the 
younger children. As noted earlier, it is interesting 
that the general pattern and size of the 
measurement error statistics for the 6th through 
12th graders is consistent with that for parents. 
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Table 6. — Estimates of gross difference rates, indexes of inconsistency and correlations from SS&D reinterviews of parents of 3rd to 5th graders 
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Table 7. — Estimates of gross difference rates, indexes of inconsistency, and correlations from SS&D reinterviews of parents of 6th to 12th graders 
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Table 7. — Estimates of gross difference rates, indexes of inconsistency, and correlations from SS&D reinterviews of parents of 6th to 12th graders— 
Continued 
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The response variance statistics for items with 
the 6th through 12th graders as respondents are 
shown in Table 8. As with the parents, the 6th 
through 12th graders had little problem with the 
general environment items. None of the eight 
items has a high gross difference rate, and the 
only three with high indexes of inconsistency 
have estimated percentages outside the range of 
20 to 80 percent. 

All four drug and alcohol education items have 
both high gross difference rates and indexes of 
inconsistency. This result is somewhat 
discouraging. It was hoped that students would 
be more consistent respondents than parents on 
these items since they were the recipients of the 
education. The difficulty in responding to these 
items may be associated with respondents 
interpreting what is meant by drug and alcohol 
education. In some schools, these topics are 
integrated into other courses and no special 
course is given on this topic. The respondents 
may not consistently interpret the question with 
respect to this type of education. 

The items about drug and alcohol use have low 
or moderate gross difference rates and indexes 
of inconsistency. This contrasts to the moderate 
and high rates for the parents for these items. It 
should be noted that the percentage reported by 
the parents is generally less than 20 percent. 

While the 6th to 12th graders are not more 
consistent than their parents in responding to the 
drug and aicohol education and use items, it 
should be noted that the estimated percentages 
for the respondents are not the same for the two 
types of respondents, indicating that the 
different types of respondents gave different 
answers. Because of the lack of consistency, 
further work on the drug and alcohol items is 
needed before they can be recommended for use 
in future studies. 



The questions about incidents are also subject to 
relatively high estimates of response variance. 
For these respondents, items about several 
different weapons were answered by 10 
respondents and are included in the table. These 
items were not included in Table 6 because 
fewer than 10 parents answered the questions. 
Eight of the 21 gross difference rates are high, 8 
are moderate, and 5 are low. For the 14 
estimated percentages in the range of 20 to 80 
percent, the indexes of inconsistency are high 
for 9 items, moderate for 4 items, and low for 1 
item. The incidents items with a high gross 
difference rate and a high index of inconsistency 
are about incidents of stealing, witnessing 
robbery, bullying, witnessing bullying, and 
physical assaults (SSSTEAL, SSFORSEE, 
SSBULLY, SSBULSEE, and SSATACK). Two 
of these items (SSFORSEE and SSBULSEE) 
also have high gross difference rates and 
indexes of inconsistency with the parents as the 
respondents. Two of the specific weapons items 
(SSMACE and SSSTICK) have both a high 
gross difference rate and a high index of 
inconsistency. 

Response Bias in the SS&D Component 

As mentioned earlier, the net difference rate 
computed based on the responses from the 
original and unreconciled reinterview is not an 
appropriate measure of response bias. This is 
because the two interviews were conducted 
under approximately the same conditions, so 
there is no reason to assume the unreconciled 
reinterview response is more accurate than the 
original response. However, the net difference 
rate computed based on the original and 
reconciled reinterview response may be a better 
measure of bias because the reconciliation 
process could result in a more accurate 
response. 



O Table 8. — Estimates of gross difference rates, indexes of inconsistency, and correlations from SS&D reinterviews of 6th to 12th graders 
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Table 8.— Estimates of gross difference rates, indexes of inconsistency, and correlations from SS&D reinterviews of 6th to 12th graders-Continued 
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Tables of net difference rates were prepared for 
each item in the SS&D with 10 or more 
respondents in the reinterview. Since the 
reinterview and reconciled values were generally 
obtained in the same interview, the sample size 
for the unreconciled and reconciled responses are 
equal. The statistics were computed for each 
item and tabulated by the four groupings of 
items. The net difference rates were computed 
using the original and unreconciled reinterview 
responses and also using the original and 
reconciled reinterview responses. 



Table 9 is a summary of the net difference rates 
based on the original and reconciled reinterview 
responses for the different types of respondents 
and groupings of items. The table gives the 
number of items in the grouping, the median net 
difference rate, and the number of items that had 
/-statistics greater than 2.0. The number of /- 
statistics greater than 2.0 is presented to 
approximate the number of items in which the 
net difference rate estimates the bias to be 
statistically significant. Because a large number 
of /-statistics are computed, about 5 percent of 
the items are expected to have /-statistics greater 
than 2.0 merely due to random fluctuation. 



Table 9. — Summary of net difference rates based on original and reconciled reinterview responses from 
the SS&D, by groups of items 



Type of respondent and item 


Number of 
estimates 


Median net 
difference rates 


Number of items 
with / greater than 
2.0 


Parents of 3rd through 5 th graders 








General environment items 


14 


0.1 


2 


Drug and alcohol education items 


5 


5.1 


2 


Incidents 


10 


6.3 


3 


Parents of 6th through 12th graders 








General environment items 


16 


0.0 


1 


Drug and alcohol education items 


5 


2.6 


2 


Drug and alcohol use items 


4 


4.5 


0 


Incidents 


15 


-2.4 


1 


6th through 12th graders 








General environment items 


8 


0.1 


2 


Drug and alcohol education items 


4 


1.9 


0 


Drug and alcohol use items 


3 


1.7 


1 


Incidents 


21 


0.8 


3 



v- 



SOURCE: U.S. Department of Education, National Center for Education Statistics, National Household Education Survey (NHES), 
spring 1993. 



The median net difference rates are generally 
close to zero, indicating that the response bias, as 
measured by the net difference rate, is not very 
large. The number of items with /-statistics 
greater than 2.0 is larger than expected by 
chance; 17 of the 105 items have /-statistics was 
greater than 2.0, while only about 5 would be 



expected to be this large by chance. There is no 
pattern in these statistics to suggest that a certain 
type of respondent or grouping of items is more 
subject to response bias. These results show that 
either that the items were not subject to large 
response bias or that the reconciliation process is 
not capturing the “true” values for the 
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respondents. Either of these situations would 
result in net difference rates such as those 
summarized in Table 9. 

Although the net difference rates computed from 
the original and unreconciled reinterview 
responses should not be used to estimate 
response biases, they are useful in examining the 
validity of the gross difference rates as a measure 
of response variances. As discussed earlier, the 
gross difference rate is a valid measure of 
response variance if the expected value of the net 
difference rate based on the original and 
unreconciled responses is zero. 

For each of the estimates included in the SS&D 
reinterview, /-statistics were computed for the 
net difference rates based on the original and 
unreconciled reinterview responses to examine 
this assumption. Of the 105 items, 15 have /- 
statistics greater than 2.0. Again, this is more 
than would be expected by chance alone, but it is 
not an obvious indication that the reinterviews 
were conducted under different conditions than 
the original. In terms of direction of bias, 9 of 
the net difference rates were negative and the 
other 6 were positive, indicating that the 
reinterview responses were neither 
systematically larger nor smaller than the 
original estimates. This lends support to the 
assumption that the same general conditions 
existed for the original and the reinterview and 
that gross difference rates should be appropriate 
measures of response variance. 

The net difference rates can also be used to 
evaluate a hypothesis that arose during the 
testing of the instruments before the survey was 
begun. Adult participants in cognitive laboratory 
sessions in the pretest said that they spoke with 
their children after completing the interview and 
realized that some of their answers were 
incorrect. It was hypothesized that the same 
might happen in the full survey, and the parents’ 
reinterview responses might be subject to greater 
differences than those of the 6th through 12th 
graders. The other observation from the 
cognitive laboratories was that the parents gave 



more positive responses after discussing the 
survey with their children. Thus, it was 
conjectured that parents might learn as a result of 
the interview and that would lead them to report 
a larger number of events in the reinterview. 

The net difference rates computed from the 
original and unreconciled reinterview responses, 
however, do not support this hypothesis (tables 
10 and 11). There are five /-statistics greater 
than 2.0 for both the parents of 6th through 12th 
graders and for the 6th through 12th graders 
themselves. For the incidents items, the net 
difference rates with /-statistics greater than 2.0 
for both types of respondents were negative, 
suggesting that both types of respondents tended 
to report more incidents in the reinterview. 
Although this evidence is not conclusive, it does 
show that the interview may not have caused the 
parents to investigate the topics covered in the 
survey and to learn new information that 
changed their responses about those topics in the 
reinterview. 

Estimates of the net difference rates for each 
item in the SS&D are given in Tables 10, 11, 
and 12. The tables include the estimated net 
difference rates, their standard errors, and the /- 
statistics for net differences computed using the 
original and the unreconciled reinterview 
responses and the original and the reconciled 
reinterview responses. 

Response Variability and Bias in the SR 
Component 

The response variance statistics, the gross 
difference rate and index of inconsistency, for 
the items from SR component of the NHES:93 
are given in Table 13. The groupings used for 
presenting the statistics in this table are items 
about the child’s development, general topics, 
reading and meals, and television viewing. 

In general, the gross difference rates and indexes 
of inconsistency (where reliable) are in the low 
to moderate range for most of the items, 



Table 10— Estimates of net difference rates from SS&D reinterviews of parents of 6th to 12th graders 
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Table 10. — Estimates of net difference rates from SS&D reinterviews of parents of 6th to 12th graders— Continued 
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Table 11. — Estimates of net difference rates from SS&D reinterviews of parents of 3rd to 5th graders 
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SOURCE: U.S. Department of Education, National Center for Education Statistics, National Household Education Survey (NHES), spring 1993. 



Table 12. — Estimates of net difference rates from SS&D reinterviews of 6th to 12th graders 
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Table 12. — Estimates of net difference rates from SS&D reinterviews of 6th to 12th graders—Continued 
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indicating the estimates from the survey were 
reported consistently in the interview and 
reinterview. All of the gross difference rates for 
the 18 development items are either low or 
moderate, and the one item with a high index of 
inconsistency has an estimated percentage 
outside the range of 20 to 80 percent. Only 2 of 
the 32 general topic items have both a high gross 
difference rate and index of inconsistency 
(SATEACHR and HNHEALTH). The first of 
these questioned whether the child said he/she 
liked the teacher and the other asked about the 
general health of the child. 

The items about reading and meals differed from 
most other items in the reiriterview in that the 
respondent was asked about activities in the past 
week. Since the child’s activities might have 
been different in the week preceding the' original 
and the reinterview, it was expected that the 
consistency of reporting these items might be 
inflated due to this factor. The results in 
Table 13 show that this may have occuired; the 
gross difference rates and indexes of 
inconsistency for many of these items are higher 
than for the other items in the table. Since the 
gross difference rate and index of inconsistency 
include a component due to the difference in the 
reference week, the statistics do not measure 
response variance well for these items. This was 
the reason that most items asking about events in 
a specific week were not selected for the 
reinterview. 

The temporal nature of the questions may have 
also affected the responses to the questions that 
asked if teachers had commented about the child 
since the beginning of the school year. These are 
the 13 items in the general topics grouping that 
have variable names that begin with the letters 
“TE.” These items are the only reinterview 
items with specific reference periods other than 
the reading and meals items. None of the 13 
items had a high gross difference rate and about 
half (7) were moderate. The indexes of 
inconsistency were high for several of the items, 
but the estimated percentages for the items are 



outside the range of 20 to 80 percent. The effect 
of the different reference period for these items 
is not as serious as for the items about reading 
and meals, probably for two reasons. First, the 
amount of change that could be expected in the 
teacher comments items is much less than in the 
reading and meals items. Second, the reference 
period for the teacher comments items is much 
longer (from the beginning of the school year) 
than for the reading and meals items (last week). 

In the NHES:91 reinterview the Head Start items 
were identified as being reported inconsistently, 
with moderate or high gross difference rates and 
indexes of inconsistency for many of the items 
(Brick and West 1992). As a result, the items 
were revised for the NHES:93 and the new items 
asked more directly about Head Start 
participation. The items in the NHES:93 
(HEADSTRT and HEADEVR) have low gross 
difference rates. These results show that the 
reporting was more consistent using the 
NHES:93 format than the NHES:91. 

The net difference rates for items included in the 
SR reinterview are given in Table 14. Only 17 
of the items were reconciled, so the number of 
these items for analysis of response bias is 
limited. Six of the 17 net differences using the 
reconciled responses have /-statistics greater than 
2.0 in absolute value, but 2 of the 6 are for items 
with different reference periods (READTO and 
READTON) so that little can be generalized 
about the meaning of the results. The net 
difference rates are all generally small with no 
estimates of bias as large as 4 percent. 

Several items in the SR interview appear in each 
NHES component concerning young children. 
These include items related to family members 
reading to children and items associated with 
children’s participation in center-based early 
childhood programs, including Head Start, day 
care centers, nursery schools, preschools, and 
prekindergartens. Items associated with reading 
in the previous week (READTO, READTON, 
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Table 13— Estimates of gross difference rates, indexes of inconsistency, and correlations from SR reinterviews 
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Table 13— Estimates of gross difference rates, indexes of inconsistency, and correlations from SR reinterviews-Continued 
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READDAY) have gross difference rates ranging 
from low to high, but have uniformly high 
indexes of inconsistency. The item concerning 
reading to the child every day in the previous 
week is high on both measures. Items on center- 
based programs, HEADSTRT and PREKIND, 
have low gross difference rates; HEADSTRT has 
a moderate index of inconsistency, whereas that 
for PREKIND is low. 

As discussed above for the SS&D, the net 
difference rates computed from the original 
interview and unreconciled reinterview responses 
in the SR can be used to assess the assumption 



that the two interviews were conducted under the 
same general conditions. Of the 55 items in the 
reinterview, the /-statistics for 15 items are 
greater than 2.0. This is nearly one-quarter of the 
items and is greater than the 5 percent that would 
be expected by chance alone. There does not 
appear to be a pattern to the estimated biases; 6 
are positive and 9 are negative. These results are 
not very supportive of the assumption that 
conditions for the original and reinterview were 
the same; rather, they raise concerns about how 
valid the gross difference rates are as measures of 
response variance. 
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Table 14. — Estimates of net difference rates from SR reinterviews 
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Table 14. — Estimates of net difference rates from SR reinterviews— Continued 
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SOURCE: U.S. Department of Education, National Center for Education Statistics, National Household Education Survey (NHES), spring 1993. 



Composite Variables 

The television viewing grouping is the only set 
of items from the SR not discussed above. This 
analysis was delayed because none of the six 
items is of much substantive interest alone. 
Rather, the responses to the items can be 
combined to form a composite variable, the 
number of hours spent watching television 
during a week, which is of great interest. A 
second composite variable, a developmental 
score based on the responses to the 
developmental items, is also examined below. 

In order to evaluate the reliability of composite 
variables or scale scores using reinterview data, 
all of the items included in the new variable had 
to be asked in the reinterview. When the 
reinterview for the NHES:93 was being planned, 
all of the items associated with the television 
viewing composite variable and the 
developmental composite variable were included 
to make this type of analysis possible. 

Two of the purposes of developing scales or 
composite variables are to obtain more reliable 
measures than can be obtained by asking one 
question and to describe a phenomenon with 
more than one dimension using a single statistic 
(Mclver and Carmines 1981). The possibility of 
improving the reliability was the prime 
motivation for having multiple television items 
that could be combined into one variable. 
Previous experience with a single item on 
television viewing had low reliability, and it was 
hoped the composite would be more reliable. 
For the SR developmental scale, items tapping 
different dimensions related to a child’s 
development were included in the interview with 
the hope that a scale based on these items would 
provide a more complete and reliable description 
of the child’s development than would be 
obtained by examining the items one at a time. 

The items about television viewing asked parents 
how many hours the child watched television 
during particular periods of the day and week. 



0 




The indexes of inconsistency for the six 
television viewing items (TVBFOR8H, 
TV8T03H, TV3DINH, TVAFDINH, TVSATH, 
and TVSUNH) are given in Table 13. The 
indexes of inconsistency are high for all six 
items. Based on the responses to these six items, 
the total number of hours per week spent 
watching television was computed, and the index 
of inconsistency was computed for this 
composite variable. The index of inconsistency 
for the composite is 22 percent, which is 
moderate. The index for the weekly composite is 
lower than all of the individual indexes, 
suggesting the desired result of constructing a 
more reliable composite variable was achieved. 

The developmental scale score was computed 
from the 18 items in the development grouping 
that were asked of parents of preschool children 
in the SR. The scale was formed by adding 
together all the items the child was able to 
perform according to the parent’s responses. 
Thus, if a parent said the child was able to 
perform half of the 18 tasks, then the scale score 
for the child was 9. The indexes of inconsistency 
for all of the 18 separate items are shown in 
Table 13. For the 8 of these items in the range 
between 20 and 80 percent, the indexes for the 
developmental items varied from 23 percent to 
43 percent, with half of the items having an 
index above 29 percent. The index of 
inconsistency for the developmental scale score 
was only 13 percent, which is less than the 
lowest of the separate items. The response 
variance for this scale is low. Again, the finding 
shows that the composite scale is more reliable 
than the separate items. 

Reconciliation Issues 

In the discussions above, some concerns have 
been raised about the validity of the estimates as 
measures of response variance and bias. These 
methodological issues and assumptions are 
addressed in this section. If the original and 
reinterview responses were not identical, the 
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respondent was asked which of the two different 
responses was correct during reconciliation. 
Many, but not all, items in the reinterview were 
reconciled. If the original and reinterview 
responses were independent and identically 
distributed and the reconciled reinterview 
response was the “true” value, then the 
reconciled responses should agree with the 
original responses about half the time and with 
the unreconciled reinterview responses about 
half the time. As discussed earlier, the 
assumption of independent, identically 
distributed responses is needed to prove that the 
gross difference rate and the index of 
inconsistency are valid measures of response 
variance. As will be shown, the results of the 
reconciliation also have some implications about 
the validity of the net difference rate as a 
measure of response bias. 

To evaluate the validity of the assumption that 
the responses had the same distribution, the data 
from the NHES:93 reinterview were classified 
into one of four possible outcomes: 

• Original and reinterview responses 
concurred (no reconciliation); 

• Reconciliation response agreed with original 
response; 

• Reconciliation response agreed with 

reinterview response; and 

• All three answers were different. 

Hubbard and Brenner (1996) report that many 
respondents do not verbalize affective reactions 
to the request to reconcile inconsistent responses. 
In addition, they observed that many respondents 
devote little time to this cognitive task 
"immediately resolving the discrepancy" (p.13). 

The fourth possibility was very unusual, given 
the wording of the reconciliation. In fact, it only 
occurred in a few cases when the respondent 
realized that neither of the two answers given 



was correct and that the questions had more than 
two possible response categories. 

The estimated percent agreement and the 
agreement ratio for each item was computed for 
the three types of SS&D respondents and the SR 
respondent and are given in Tables 15, 16, 17, 
and 18. The agreement ratio is used to measure 
whether the responses to the reconciliation were 
more likely to agree with the original or 
unreconciled reinterview response. If the 
agreement ratio is greater than 1 .0, then more of 
the reconciled responses agreed with the 
unreconciled reinterview responses than with the 
original responses. 

In general, the reconciled responses were more 
likely to agree with the responses given in the 
reinterview rather than in the original interview. 
For the parents of 6th through 12th graders, 35 of 
the 38 items had agreement ratios greater than 1 
(one item did not require reconciliation); for 
parents of 3rd through 5th graders, 17 of 28 
items had agreement ratios greater than 1 (two 
items did not require reconciliation); for 6th 
through 12th graders, 27 of the 30 items had 
ratios of greater than 1 (the 6 items with 10 
observations were dropped for this analysis); and 
for the SR parent respondents, 16 of 18 of the 
reconciled items had ratios of greater than 1. 

These findings are consistent with those from the 
NHES:91 reinterview, where the reconciled 
responses agreed with the unreconciled 
reinterview responses three times more often than 
with the original response (Brick and West 1992). 
In that reinterview, the respondents were told 
which answer they had just given in the 
reinterview and which was the original response. 
Since giving the respondents this information 
might influence their choice, the procedure was 
changed for the NHES:93 reinterview and 
respondents were not told which response was 
from the original interview and which was from 
the reinterview. Despite the revisions, the 
respondents were more likely to state that the 
reinterview response was the correct one. 



Table 15. — Estimates of agreement between original, unreconciled reinterview, and reconciled reinterview for SS&D, parents of 6th to 12th graders 
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Table 15. — Estimates of agreement between original, unreconciled reinterview, and reconciled reinterview for SS&D, parents of 6th to 12th graders— 
Continued 
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Table 16. Estimates of agreement between original, unreconciled reinterview, and reconciled reinterview for SS&D, parents of 3rd to 5th graders 
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Table 17. — Estimates of agreement between original, unreconciled reinterview, and reconciled interview for SS&D, 6th to 12 graders 
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Table 17.— Estimates of agreement between original, unreconciled reinterview, and reconciled interview for SS&D, 6th to 12 graders-Continued 
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Table 18.— Estimates of agreement between original, unreconciled reinterview, and reconciled reinterview for SR 
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The literature on the cognitive theory associated 
with this type of behavior is not extensively 
developed, perhaps because reconciliation of 
responses is not frequently attempted. One 
theory that may be pertinent is related to the 
way people recall events and is called the 
consistency principle. Pearson et al. (1992) 
suggest “when the consistency principle is 
invoked, people tend to view their current 
standing on an attribute as an accurate reflection 
of their past status.” If this principle holds in 
the reinterview context, then it is likely that 
people will tend to agree with their latest 
response (the reinterview response) and assume 
the earlier (original) response was either 
incorrectly recorded or that some other type of 
error occurred. 

If the consistency principle applies and 
respondents were trying to be consistent with 
their most recent responses, then the net 
difference rates should not be used to estimate 
response bias. This follows because net 
difference rates computed from reconciled 
values are measures of response bias only if the 
reconciled responses are more accurate than the 
original interview responses. If the reconciled 
response is motivated more by the need to give 
consistent rather than accurate responses, then 
there is little basis for the claim that the 
reconciled response is closer to the “true” 
value. While this explanation undermines the 
value of the net difference rate as a measure of 
bias, it is consistent with theoretical framework 
required to support the gross difference rate and 
index of inconsistency as a measures response 
variance. 

On the other hand, it is possible to explain these 
results with another hypothesis. Suppose 
respondents thought about what they said in the 
original interview and recalled things they failed 
to say at the time of the original interview. 
Alternatively, suppose respondents were willing 
to give more accurate responses in the 
reinterview because they became more 
comfortable with the interview process as a 



result of the original interview. If either of 
these happened, then the reinterview responses 
should be more reliable than the original 
interview responses. In these circumstances, the 
agreement ratios should be greater than 1 
because the reinterview responses are more 
reliable. Furthermore, the net difference rates 
using the reconciled reinterview responses could 
be useful measures of response bias because 
they might be better reflections of the “true” 
value than the original responses. However, the 
gross difference rates would no longer be valid 
measures of response variance because the 
situation described is not consistent with having 
independent, identically distributed responses 
from both interviews. 

No empirical data are available to determine 
whether either of these explanations is correct. 
However, the NHES:93 reinterview procedures 
were developed to support estimating response 
variance rather than response bias. The same 
pool of interviewers was used for both 
interviews, the same questions were asked, and 
no additional guidance was given to the 
respondents or interviewers to support 
estimating response bias. As a result, the first 
hypothesis appears more plausible. If the first 
hypothesis holds, then the gross difference rates 
and indexes of inconsistency should be 
approximately valid measures of response 
variance and the net difference rates should be 
poor estimates of response bias. 

Summary and Recommendations 

The NHES:93 reinterview study provides a 
great deal of information on the reliability of the 
estimates from the survey and is a valuable 
source for examining the reliability of specific 
items. The gross difference rate and index of 
inconsistency are used to measure the 
consistency of response, while the net difference 
rate, computed using the original and reconciled 
reinterview responses, is used to assess response 
bias. 



Many of the items in the SS&D had gross 
difference rates and indexes of inconsistency in 
the moderate to high range. The general 
environment items had low to moderate gross 
difference rates and indexes of inconsistency, 
but items that referred to specific events had 
larger estimates of response variance. For 
example, the gross difference rates and indexes 
of inconsistency were higher for estimates about 
stealing (a specific event) than estimates about 
the presence of metal detectors in the school 
(general environment). Several possible 
explanations were proposed for the lower 
reliability of these items but no evidence from 
this study can be used to evaluate these possible 
explanations. Cognitive laboratory work should 
be considered for these items prior to their 
inclusion in future surveys. 

The items in the SS&D about alcohol and drug 
education and usage also had moderate to high 
response variance statistics. It was conjectured 
that respondents may have had some difficulty 
comprehending the intent of these items. The 
respondents may have interpreted what was 
being requested differently in the interviews. 
Additional developmental work is needed for 
these items before they are used in subsequent 
surveys. 

Another important finding from the SS&D 
component reinterview was that the reliability 
of the items was approximately the same for 
both 6th to 12th graders and their parents. 
Initially, there was some concern that youth 
would be poorer respondents than adults in a 
telephone interview. The gross difference rates 
and indexes of inconsistency for the two types 
of respondents were nearly the same. While 
there is reason for some concern about the 
reliability of some items specific to the SS&D 
interview, the finding that adults and youth do 
not differ substantially in this regard should 
allay some of the concerns about the ability of 
youth to respond to a telephone survey like the 
NHES. 



In general, the gross difference rates and 
indexes of inconsistency from the SR 
reinterview showed that the items were reported * 
consistently. The gross difference rates and 
indexes for most of the items were in the low to 
moderate range for the SR items. Only two 
items, as discussed in the body of the report, had 
response variance statistics that warrant further 
evaluation. An important finding was that the 
revised method of asking about participation in 
Head Start programs resulted in more consistent 
responses than the method used in the 
NHES:91. 

Another important finding from the SR 
component concerned the reliability of the 
television viewing composite variable and the 
developmental scale. The evaluation of the 
response variability for these two variables was 
very promising. The indexes of inconsistency 
for the variables were much lower than most of 
the indexes for the items used in creating the 
composites. This finding is encouraging for 
future analytic development and suggests that 
even difficult concepts may be attacked by 
using a set of related items rather than a single, 
complex one. Multiple questions do involve 
more interview time and data collection cost, 
but for key topics this seems a modest price for 
more reliable estimates. The NHES is well 
suited for this way of exploring complex issues, 
since its indepth topical components provide an 
opportunity to include multiple indicators of a 
construct, an approach that is not possible in the 
more limited space available in supplements to 
other household surveys or in general social 
surveys. 

A final topic concerns a methodological issue. 
As discussed in the last section, the analysis of 
the reconciled responses yielded mixed results. 
One way of interpreting the findings is that the 
net difference rates computed using the 
reconciled responses are not valid estimates of 
response bias because the respondents are 
giving responses that are very consistent with 
the reinterview rather than giving more accurate 



responses. If the reconciled responses are not 
valid estimates of response bias, then the use of 
reconciliation needs to be reconsidered. One 
alternative to the type of reconciliation used in 
the NHES:93 is a more indepth evaluation for a 
selected number of items if the original and 
reinterview responses differ. Special efforts 
could be used so that the final responses could 
be considered more valid than the original 
responses. An intensive reinterview study was 
conducted for the Adult Education component 
of the NHES:95, in which a flexible interview 
protocol was used to explore response issues 
using techniques similar to those used in 
cognitive laboratory debriefing (Brick et al. 
1996). A different approach to intensive 
reinterviewing was carried out in the 1991 
Teacher Follow-up Survey (Jenkins and Wetzel 
1994). These types of reconciliation could 



clarify some of the ambiguities resulting from 
the current procedure. 

Analysts using the NHES:93 data should find 
the results of this reinterview effort valuable 
when using the data sets. In particular, users 
should be careful in drawing conclusions based 
on the items with indexes of inconsistency 
greater than 45 and gross difference rates of 
greater than 20. High response variability, as 
indicated by these measures, weakens the 
relationships between variables, for example, 
attenuating the correlations between measures. 
Researchers may also want to take this research 
into account when planning studies of their own, 
with these items. Further examination of items 
that tend to have response problems may be 
warranted. 
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Appendix A 



Reinterview Questionnaires 



NHES:93 School Safety and Discipline Reinterview 



INTRO. Hello, my name is (interviewer). A few weeks ago, someone from our staff conducted an interview 
with you about (child) for the U.S. Department of Education. We are reasking selected 
questions from the interview as a measure of survey quality. These questions should only take 
a few minutes. 

|| If Respondent /s a youth, g o to YIN TRO. 

*P 1 o. Does (child) go to a public or a private school? 

SCPUBUC 

PUBLIC 1 

PRIVATE 2 

REFUSED - 7 

DON’T KNOW -8 



(GO TO P1 1) 
(GO TO PI 3) 
(GO TO P1 1) 
(GO TO P1 1) 



*P11. 

SCASSIGN 



Is it (his/her) regularly assigned school or a school that you chose? 



ASSIGNED 1 (GO TO P 1 7) 

CHOSEN 2 ( G0 TO PI 3) 

ASSIGNED SCHOOL IS SCHOOL OF CHOICE 3 (GO TO P17) 

REFUSED -7 (GOTO PI 7) 

DON’T KNOW *8 (GO TO P 1 7) 



*P13. 

SCREASON 



What is the main reason (child) goes to this school and not some other school? 
[probe: What is the most important reason?] 



FOR SPECIAL COURSES AND/OR ACADEMIC 

PROGRAMS, E.G., A MAGNET SCHOOL 1 

FOR SPECIAL NONACADEMIC PROGRAMS, E.G., 

AFTER SCHOOL CARE OR SPORTS 2 

EXPELLED FROM ANOTHER SCHOOL 3 

SENT TO ALTERNATIVE SCHOOL 4 

BETTER ACADEMICALLY 5 

SMALLER SCHOOL/CLASSES 6 

SAFER SCHOOL/AREA 7 

MORE CONVENIENT LOCATION 8 

BETTER DISCIPLINE IN THIS SCHOOL 9 

RELIGIOUS REASONS 10 

CANNOT AFFORD ANOTHER SCHOOL 1 1 

OTHER (SPECIFY) , 91 

REFUSED *7 

DON’T KNOW -8 



*P1 7 . is this the first year (child) has attended this school? 

SCFIRST 

YES 1 

NO 2 

REFUSED *7 

DON'T KNOW -8 
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*P18. 

scstud [probe: 



SCSTUDGR 

*P19. 

SCSAMBTH 



YINTRO. 

*PY21. 

SBCHALNG 

SBPRIDIS 

*P25. 

SBMISBBH 



o 




Approximately how many students are enrolled in (child's) school? Would you say... 
Do you know the number in (his/her) grade ?] 



Under 300... .1 

300-599 2 

600-999, or 3 

1,000 or more? 4 

NUMBER OF STUDENTS IN GRADE GIVEN 5 

REFUSED -7 

DON’T KNOW -8 



If PI 8 = 5, overfay number of students in grade. 



Number of students in grade 

Approximately what percent of the students are of the same race or ethnic background as 
(child)? Would it be... 



Less than 25 percent 1 

25 to 75 percent, or 2 

More than 75 percent? 3 

refused -7 

DON’T KNOW -8 



[if respondent is A youth]: Hello, this is (interviewer). A few weeks ago, someone from our 
staff conducted an interview with you for the U.S. Department of Education. We are reasking 
selected questions from the interview as a measure of survey quality. These questions should 
only take a few minutes. 



When you think about [(CHiLD's)/your] experiences at [(his/her)/your] school since the beginning 
of this school year, would you strongly agree, agree, disagree, or strongly disagree with each of 
the following statements? 

[1 = STRONGLY AGREE; 2 = AGREE; 3 = DISAGREE; 4 = STRONGLY DISAGREE] 



a. [(child) is/l am] challenged at school 

e. The principal and assistant principal maintain 
good discipline at [(CHiLD's)/my] school 



SA A D SD R DK 

1 2 3 4 -7 -8 

1 2 3 4 -7 -8 



Has misbehavior by students in (child's) class this year interfered with (his/her) opportunity to 
learn? Would you say... 



A lot 1 

Somewhat 2 

A little, or 3 

Not at all? 4 

refused -7 

donTknow -8 
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*PY26. (Have you heard/Do you know) of any of the following things happening during this school 

year ? Things being stolen from lockers or desks? 

SSSTEAL 

YES 

NO 

REFUSED 

DON'T KNOW 

*PY28. Did it happen to [(CHiLD)/you) this school year? 

SSSTEYOU 

* YES 

NO 

REFUSED 

DON'T KNOW 



.1 

.2 

-7 

-8 



.1 (GO TO PY28) 

.2 (GO TO PY29) 

-7 (GO TO P Y29) 

-8 (GO TO PY29) 



*PY29. 

SSFORCE 



*PY30. 

SSFORSEE 



(Have you heard/Do you know) of money or other things being taken directly from students or 
teachers by force or threat of force at school or on the way to or from school this school year? 



YES 1 (go to PY30) 

NO : 2 (GO TO PY34) 

REFUSED -7 (GO TO PY34) 

DON'T KNOW i-8 (GO TO PY34) 

Did [(CHiLD)/you) see an incident like this happen to someone else? 

yes 1 

no 2 

REFUSED -7 

DON’T KNOW -8 



*PY32. Did it happen to [(CHiLD)Zyou] this school year? 



SSFORYOU YES 1 

NO 2 

REFUSED -7 

DON'T KNOW -8 



*PY34. 



SSBULLY 



(Have you heard/Do you know) of any incidents of bullying during this school year? For 
example, do some students pick on others a lot or can they make other students do things like 
give them money? 



yes 1 (go to PY35) 

NO 2 (GO TO PY39) 

REFUSED -7 (GO TO PY39) 

DON’T KNOW -8 (GO TO PY39) 



*PY35. Did [(CHiLD)Zyou] see an incident like this happen to someone else? 



SSBULSEE 

YES 1 

NO 2 

REFUSED -7 

DON'T KNOW -8 
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*PY37. Did it happen to [(CHiLD)/you] this school year? 



SSBULYOU 

YES ... 1 

NO 2 

REFUSED... -7 

DON'T KNOW ...-8 



*PY39. 

SSATTACK 



*PY40. 

SSATTSEE 



*PY42. 

SSATTYOU 



(Have you heard/Do you know) of any students or teachers being physically attacked, or 
involved in fights, during this school year? 



yes 1 

no 2 

REFUSED -7 

DON'T KNOW -8 



(GOTOPY40) 

(GO TO BOX AFTER PY42) 
(GO TO BOX AFTER PY42) 
(GO TO BOX AFTER PY42) 



Did [(CHiLD)/you] see an incident like this happen to someone else? 



yes ; .1 

no 2 

REFUSED...., -7 

DON'T KNOW -8 



Did it happen to [(CHiLD)Zyou] this school year? 



yes.... t 

NO 2 

REFUSED i -7 

DON'T KNOW.... -8 



If Respondent Isa parent, go to box after Y44. 



Y44. 


Did you do any of the following things because you were worried that someone might hurt i 




bother you? 














YES 


n6 


R 


DK 


SSROUTE 


a. Take a special route to get to school? 


1: 


. 2 


-7 


-8 


SS PLACES 


b. Stay away from certain places in the school? 


1 


2 


-7 


-8 


SSPARKNG 


c. Stay away from the school parking lots or other 












places on school grounds? 




2 


-7 


-8 



If Respondent Is a youth, go to PY46. If Respondent Is a 
' parent and P26 = 1 orPY29 - 1 orPY34 = 1 orPY39 = 1, ask 
P45. Else, go to box after P45. 



Have any of those incidents that happened at (child's) school this year interfered with (his/her) 
opportunity to leam? Would you say... 



SSINCDNT 

A lot 1 

Somewhat 2 

A little, or 3 

Not at all? 4 

refused .7 

don't know -8 
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lfP6<6or (P6 = 13 or 14 and P7 < 6) and respondent Is a 
parent, go to PY55. Else, if PY26 = 1 or PY29 = 1 or PY34 = 
1 orPY39 = 1, ask PY46; else, go to PY47. 



PY46. Were any of those incidents that happened at [(CHiLD's)/your] school this year racially 



motivated? 

SSRACIAL 

YES 1 

NO 2 

REFUSED -7 

DON'T KNOW -8 



PY47. 

SSGANGS 



PY50. 

SSGANREL 



Y51. 

SSWEAYOU 



Do any of the students at [(child’s )/your] school belong to fighting gangs? 



YES 1 (GO TO BOX) 

NO 2 (GO TO BOX AFTER PY50) 

REFUSED -7 (GO TO BOX AFTER PY50) 

DON'T KNOW -8 (GO TO BOX AFTER PY50) 



If PY26 = 1 or PY29 = 1 orPY34 = 1 orPY39 = 1, ask PY50. 
Else, go to box after PY50. 



Were any of those incidents that happened at [(CHiLD's)/your] school this year related to gang 
activity? 



YES 1 

NO 2 

REFUSED -7 

DON'T KNOW -8 



If Respondent Is a parent, go to PY53. 



During this school year, did you ever bring something to school to protect yourself from being 
attacked or harmed? 



yes 1 

no 2 

REFUSED -7 

DON’T KNOW ; -8 



(GO TO Y52) 
(GOTO PY53) 
(GOTO PY53) 
(GO TO PY53) 
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Y52. 



Did you bring ... 



YES NO 



SSGUN 


a.. 


A gun 




2 


SSKNIFE 




b 




1 


SSBRASS 


c. 


Brass knuckles 




2 


SSRAZOR 


d. 


Razor blade 




2 


SSJEWLRY 


e. 


Spiked jewelry 




2 


SSMACE 


f. 


Mace 


1 


2 


SSCHUCKS 


g- 


Nunchucks 




A 

2 


SSSTICK 


h. 


A stick, club, or bat, or 


1 


2 


SSOTHER 


i. 


Something else 




2 






What was that? 






SSOTHEOS 












j. 


REFUSED 


-7 






k. 


DON'T know 







2 



(Have you heard/Do you know) of any (other) students bringing weapons into [(CHiLD's)/your] 



school this year? 

SSWEAOTH 

YES 1 

NO 2 

refused : -7 

DON'T KNOW -8 



*PY55. 



SSGUARDS 

SSMETAL 



[Do you know if (child's) school takes/Does your school, take] any particular measures to 
ensure the safety of students? For example, does the school have... 



a. 

b. 



Security guards? 
Metal detectors?. 



YES 


NO 


R 


DK 




2 


-7 


-8 




2 


-7 


-8 



IfRespondent Is a youth, go to Y6Q. 



*P56. 

SDPOUCY 



As far as you know, does (child’s) school have a written discipline policy? 



yes 

no 

REFUSED 

DON'T KNOW. 



..1 

..2 

-7 

-8 



(GO TO P59) 

(GO TO BOX AFTER P59) 
(GO TO BOX AFTER P59) 
(GO TO BOX AFTER P59) 



* p 59- Does it cover alcohol and other drug possession, use, and distribution? 



SDDRUGS 

YES 1 

NO 2 

REFUSED .7 

DON'T KNOW -8 



IfRespondent Is a parent and P6<6or (P6 = 13 or 14 and 
P7 < 6), go to P67. Efse, go to PY63. 
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Y60. 

SDFAIR 

SDENFORC 

PY63. 

TADRUNK 

PY64. 

TAHIGH 

PY66. 

TADEAL 



*P67. 

EDDRUGS 

‘PY68. 

EDPART 

EDCOURSE 

EDDEMO 

EDCLUBS 



Now I am going to read a list of statements that could describe a school. Thinking about your 
school this year, would you strongly agree, agree, disagree, or strongly disagree with the 
following... 

[1=STRONGLY AGREE; 2 = AGREE; 3 = DISAGREE; 4 = STRONGLY DISAGREE] 

SA A D SD R DK 

b. The school rules are fair 1 2 3 4 -7 -8 

d. The school rules are strictly enforced 1 2 3 4 -7 -8 



(Have you heard of/Have you seen) any students (having been) drunk or showing the effects of 
alcohol when they were at [(CHiLD's)/your] school this year? 



yes 1 

no 2 

REFUSED -7 

don't know -8 



(Have you heard of/Have you seen) any students (having been) high on other drugs such as 
marijuana, LSD, or cocaine when they were at [(child's )/your] school this year? 



yes 1 

no 2 

REFUSED -7 

DON'T KNOW -8 



(Have you heard of/Have you seen) anyone dealing drugs at school or within sight of school 
property this year? 



yes 1 

no 2 

REFUSED -7 

don't know -8 



If Respondent is a youth, go to PY68. 



Has (child) had any alcohol or other drug education course or program at school during this 
school year? 



YES 1 (GOTO PY68) 

no 2 (GO TO BOX AFTER Y68) 

REFUSED -7 (GO TO BOX AFTER Y68) 

DON'T KNOW -8 (GO TO BOX AFTER Y68) 



There are many different ways that alcohol or other drug education can be presented to 
students. Did [(CHiLD)/you] receive alcohol or other drug education in school this year... 







YES 


NO 


R 


DK 


a. 


As part of one of the regular courses, 

like science, health, or PE? 


1 


2 


-7 


-8 


b. 


A special course about alcohol or other drugs? 


1 


2 


-7 


-8 


c. 


At assemblies or demonstrations outside of 
classes? 


1 


2 


-7 


-8 


d. 


In other school activities or clubs? 


1 


2 


-7 


-8 
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if Respondent is a parent and P6<6or (PS = 13 or 14 and 
P7 < 6), go to P81. Else, go to P77. 



| If Respondent is a youth, go to PY97. || 

P77. During this school year, has (child) participated in any school activities such as sports teams, 

band or chorus, school clubs, or student government? 



CCSCHL 

YES 1 

NO 2 

SCHOOL DOES NOT OFFER ANY 3 

REFUSED -7 

DON'T KNOW -8 



*P81. How many times has (child) moved from one home or household to another during the last 5 

years? 

FCMOVED 

NUMBER 

REFUSED -7 

DON'T KNOW -8 



*P83. Would you say that you are very satisfied, somewhat satisfied, somewhat dissatisfied, or very 

dissatisfied... 

[1 = VERY SATISFIED; 2 = SOMEWHAT SATISFIED; 3 = SOMEWHAT DISSATISFIED; 

4 = VERY DISSATISFIED] 

VS SS SD VD R DK 



fcstds c. With the academic standards of the school? 1 2 3 4 -7 -8 

fcorder d. With the order and discipline at the school? 1 2 3 4 -7 -8 



*P89. 



FCMEETNG 



*PY97. 



Since the beginning of this school year, have you [or (child's) (mother/stepmother/foster 
mother/father/stepfather/foster father)] ... 



a. Attended a general school meeting, for 

example, back to school night or a meeting 
of a parent-teacher organization? 

Would you say [(CHiLD's)/your] school ... 



YES NO R DK 

..1 2 -7 -8 



coschool Is safer than your neighborhood 1 

About as safe, or 2 

Not as safe as your neighborhood? 3 

refused -7 

DON'T KNOW -8 



*P124R. Have you and (child) talked about (his/her) school experiences more than usual as a result of 

participating in this survey? 



TALKMOR2 YES 1 

NO 2 

REFUSED -7 

DON'T KNOW -8 
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*P125R. Have you learned more about (child's) school from (him/her) or from other sources since the 

original interview? 



LERNMOR2 YES. 1 

NO 2 

REFUSED.... -7 

DON’T KNOW -8 



If discrepancies need to be reconciled, ask 
RECONCILIATION. Else, go to CLOSE3. \ 

RECONCILIATION. We asked you [display full question ]. One time we recorded [display answer that 

comes first in original response categories] and one time we recorded [display answer 
that comes second in original response categories]. What is the best answer? : 

[Display all original response categories. ] 

CLOSE3. Those are all the questions I have. Thank you for your time. 
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School Readiness Reinterview 



INTRO. Hello, my name is (interviewer). A few weeks ago, someone from our staff conducted an interview 
with you about (child) for the U.S. Department of Education. We are reasking selected 
questions from the interview as a measure of survey quality. These questions should only take 
a few minutes. 



3 



The developmental profile Items are asked for preschooler, 
all others go to box before ECINTRO. 



DPINTRO. These questions are about things that different children do at different ages. These things may 
or may not be true for (child). 

R14. Can (child) identify the colors red, yellow, blue, and green by name? Would you say... 



DPCOLOR 

All of them 1 

Some of them, or 2 

None of them? 3 

refused -7 

DON’T KNOW -8 

R15. Can (he/she) recognize... 

DPLETTER 

All of the letters of the alphabet 1 

Most of them 2 

Some of them, or 3 

None of them? 4 

REFUSED -7 

DON'T KNOW -8 



R16. 

DPCOUNT 



R17. 

DPNAME 



How high can (child) count? Would you say... 

Not at all 1 



Up to five 

Up to ten 

Up to twenty 

Up to fifty, or 

Up to 100 or more? 

REFUSED 


-7 


donT know 


-A 


Can (child) write (his/her) first name, even if some of the letters are backwards? 


YES 




NO 




REFUSED 




DON'T KNOW 
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R18. Can (he/she) button (his/her) clothes? 

DPBUTTON 

YES 1 

NO 2 

REFUSED -7 

DON'T KNOW -8 

R1 9. Does (he/she) hold a pencil properly? 

DPPENCH - - 

' ■ YES 1 

NO 2 

REFUSED -7 

DON'T KNOW -8 



R20. Does (he/she) mostly write and draw rather than scribble? 



DPWRITE 

YES 1 

NO 2 

REFUSED -7 

DON'T KNOW -8 

R21 . Does (he/she) trip, stumble, or fall easily? 

DPFALL 

YES .1 

NO 2 

REFUSED -7 

DON'T KNOW -8 



R22. Can (child) be left alone with a babysitter without a big fuss? 



DPSITTER 

YES 1 

NO 2 

REFUSED -7 

DON'T KNOW -8 



R23. Does (child) often have temper tantrums? 

DPTEMPER 

YES. 1 

NO 2 

REFUSED -7 

DON'T KNOW -8 

R24. Is (child) afraid to speak to people (he/she) doesn't know? 

DPAFRADl 

YES 1 

NO 2 

REFUSED -7 

DON'T KNOW -8 



R25. Is (he/she) very restless, and does (he/she) fidget a lot? 



DPFIDGET 

YES 1 

NO 2 

REFUSED -7 

DON'T KNOW -8 
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R26. Does (he/she) have a very short attention span? 



DP ATTN 

YES 1 

NO ...2 

REFUSED -7 

DON'T KNOW -8 



R27. When (he/she) speaks, is (child) understandable to a stranger? 



DPSPEAK 

YES i 1 

NO 2 

REFUSED -7 

DON'T KNOW -8 



R28. Did (he/she) start speaking later than other children you know? 



DPSPELA 

YES 1 

NO 2 

REFUSED -7 

DON'T KNOW -8 

R29. Does (child) stutter or stammer? 

DPSTUTE 

YES 1 

NO . 2 

REFUSED -7 

DON’T KNOW -8 



R30. 

DPTV 



R31. 

DPBEND 



Does (he/she) turn on the television at a very high volume? 



yes 1 

no 2 

REFUSED ....-7 

DON’T KNOW -8 



Does (he/she) bend over to look very closely at pictures or drawings? 



yes 

no 

REFUSED 

DON’T KNOW. 



..1 

.2 

-7 

-8 



Preschoolers are asked about current participation and 
past participation if current = no. 

Kindergartners (grades T, K, P) and primary students are 
asked about participation prior to kindergarten (or first 
grade if they did not attend^ kindergarten). 



ECINTRO. These next questions are about early childhood programs and organized day care centers. We 
are not including babysitting or child care provided in private homes. 
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R32. 

HEADSTRT 



33. 

HEADEVR 



36. 



PREKIND 



37. 

PREKEVR 



R40. 

PREKANY 



[preschoolers only:] 

Is (child) now attending-of-enrOlled in Head Start? 



yes 1 

no 2 

REFUSED -7 

DON’T KNOW -8 



(GO TO R36) 
(GO TO R33) 
(GO TO R33) 
(GO TO R33) 



[Prior to starting (kindergarten/first grade), did/Has] (child) ever (attend/attended) Head Start? 



yes 1 

no 2 

REFUSED -7 

DON’T KNOW -8 



[preschoolers only:] 

(Other than Head Start) Is (child) now attending a nursery school, prekindergarten, preschool, 
or a day care center? 



yes 1 

no 2 

REFUSED -7 

DON’T KNOW -8 



(GO TO R40) 
(GO TO R37) 
(GO TO R37) 
(GO TO R37) 



[Prior to starting (kindergarten/first grade), did/Has] (child) ever (attend/attended) nursery 
school, prekindergarten, preschool, or a day care center (other than Head Start)? 



yes 1 

no 2 

REFUSED -7 

DON’T KNOW -8 



If R32 or R33 or R36 or R37 = yes, ask R40. Else, go to box 

after R40. 



Have any of the (Head Start programs) (or) (nursery schools, prekindergartens, preschools, or 
day care centers) (child) has gone to had an educational program? 



yes 1 

no 2 

REFUSED -7 

DON’T KNOW -8 



Preschoolers go to R92. 
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SAINTRO. 

R51. 



Children sometimes have difficulty adjusting to (school/kindergarten). 

On the average, during the first two months of this school year, that is, last September and 
October... 

[1 = MORE THAN ONCE A WEEK; 2 = ONCE A WEEK OR LESS; 3 = NOT AT ALL] 

[IF REFUSED -7, OR DON’T KNOW -8, GO TO NEXT INDICATOR, OR IF LAST INDICATOR, GO TO 
R52.] 



>1 WK 



1 WK< 



None 



SACOMPLA 


a. 


Did (child) complain about school more 
than once a week, once a week or less, 
or not at all? 


... 1 




2 




3 


SALEAVE 


b. 


Was (child) upset or reluctant to go to school? 


... 1 




2 




3 


SASICK 


c. 


Did (he/she) pretend to be sick to stay 

home from school? 


... 1 




2 




3 


SAGOOD 


d. 


Did (he/she) say good things about school? 


... 1 




2 




3 


SATEACHR 


e. 


Did (child) say (he/she) liked (his/her) teacher? 


... 1 




2 




3 


SASCHOOL 


f. 


Did (he/she) look forward to going to school? 


... 1 




2 




3 


R52. 


Since the beginning of this school year, has a teacher said or written that... 














YES 


NO 


R 


DK 




TEWELL 


a. 


(child) has been doing really well in school? 




i 


2 


-7 


-8 


TEABIL 


b. 


(child) has not been learning up to (his/her) 
capabilities? 




i 


2 


-7 


-8 


TEATTENT 


c. 


(child) doesn't concentrate, doesn't pay 
















attention for long? 




2 


-7 


, -8 


TEDISRUP 


d. 


(child) has been acting up in school or 
















disrupting the class? 


i 


2 


-7 


-8 


TESAD 


e. 


(child) has often seemed sad or unhappy in class? 


i 


2 


-7 


-8 


TEFIDGET 


f. 


(child) has been very restless, fidgets all the time, 
















or doesn’t sit still? 




2 


-7 


-8 


TESHARE 


g 


(child) has been having trouble taking turns, sharing, 
















‘ or cooperating with other children? 


i 


2 


-7 


-8 


TEGROUP 


h. 


(child) gets along with other children or works 
















well in a group? 




2 


-7 


-8 


TEENTHUS 


i. 


(child) is very enthusiastic and interested in a 
















lot of different things? 




2 


-7 


-8 


TENONEW 


j- 


(child) lacks confidence in learning new things 
















or taking part in new activities? 


i 


2 


-7 


-8 


TECLEAR 


k. 


It's hard to understand what (child) is saying? 


i 


2 


-7 


-8 


TESLEEPY 


1. 


(child) was often sleepy or tired in class? 


i 


2 


-7 


-8 


TEEXPRES 


m. 


(child) likes to speak out in class and express 
















(his/her) ideas? 


i 


2 


-7 


-8 






|| Primary students go to R79. Kindergartners go to R92. 










R79. 


Has (child) received any special help in school this year for children who are having trouble 




with... 




















YES NO 


R 


DK 




PREADING 


a. 


Reading? 


i ; 


2 


-7 


-8 




PMATH 


b. 


Arithmetic? 




-7 


-8 




PADJUST 


c. 


Adjusting to school? 


1 2 


-7 


-8 




PSPEECH 


d. 


Speech? 




-7 


-8 




PENGUSH 


e. 


English as a second language? 


1 2 


-7 


-8 
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R92. On average, about how many hours of television or video tapes does (child) watch at home 

each weekday, that is, Monday through Friday? How about... 

[ENTER 95 IF R DOES NOT HAVE TV, GO TO R96.] 

R DK 



TVBFOR8H 


a. 


Before 8 am?...! 


Hours 


Min. 


-7 


-8 


TVBFOR8M 

TV8T03H 


b. 


Between 8 am and 3 pm? 


Hours 


Min. 


-7 


-8 


tv8to3m2 

TV3TDINH 


c. 


Between 3 pm and dinner time? 


Hours 


Min. 


-7 


-8 


TV3DINM 

TVAFDIH 


d. 


After dinner time? 


Hours 


Min. 


-7 


-8 



TVAFDIM 



R93. 



TVSATH 
TVS ATM 
TVS UN H 
TVSUNM 



How about on Saturday and Sunday? How many hours does (child) watch television or video 
tapes at home on... 











R 


DK 


a. 


Saturday? 


Hours 


Min. 


-7 


-8 


b. 


Sunday? 


Hours 


Min. 


-7 


-8 



R96. 

READTO 



R97. 

READTON 



R98. 

READD AY 



Now I'd like to talk with you about activities in your home in the past week. 
In the past week, have you or has someone in your family read to (child)? 



yes 

no 

REFUSED 

DON'T KNOW. 



.1 

.2 

-7 

-8 



(GO TO R97) 
(GOTO R1 06) 
(GOTO R1 06) 
(GOTOR106) 



How many times? Would you say... 



One or two times, or 

Three or more? 

REFUSED 

DON'T KNOW 



.1 (GOTOR106) 
.2 (GO TO R98) 
-7 (GOTO R1 06) 
-8 (GOTOR106) 



Was that every day in the past week ? 



yes 1 

no 2 

REFUSED -7 

DON’T KNOW -8 



R106. In general, would you say that (child’s) health is... 



HNHEALTH 

Excellent 1 

Very good 2 

Good 3 

Fair, or 4 

Poor? 5 

refused -7 

DON'T KNOW -8 
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R110. 



HNDOCWHN 



About how long has it been since (child) last saw a medical doctor or other health professional 
for a checkup, shots, or other routine care? Would you say... 



Less than 1 year 

1 year, but less than 2 years 

2 years or more? 

refused 

DON’T KNOW 



.1 ' 

.2 

.3 

-7 

-8 



R113. During the last week, that is, since last (day of week), on how many days did (child) eat 

breakfast, either at home or somewhere else? 



HNBREAK 

DAYS . 0 

REFUSED -7 

DON’T KNOW -8 



| If child Is a preschooler, go to R115. Else, go to R118. 1 

R 1 1 5 . During the last week, on how many days did the whole family sit down to eat dinner together? 



HNDINNER 

DAYS 0 

REFUSED -7 

DON’T KNOW -8 



If child is In school or kindergarten, or If child Is a 
preschooler In a center-based program, go to R118. Else, 
go to R159. 



R118. Does (child) receive government funded free or reduced price breakfast or lunch at 



[(PROGRAM)/SChOOl]? 

HNFREE 

YES 1 

NO .2 

REFUSED -7 

DON’T KNOW -8 



Kindergartners and primary students, go to CLOSE3; 
Preschoolers, go to R159. 



ARINTROR. I have a few more general questions about issues involving children. 
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Now I'm going to ask you how important you think it is for any child to know or do certain things 
to be ready for kindergarten. 

How important do you think it is that a child... 









E 


VI 


SI 


NVI 


Nl 


R 


DK 


KPCOUNT 


a. 


Can count to 20 or more? Would you say 




















essential, very important, somewhat important, 




















not very important, or not at all important? 


1 


2 


3 


4 


5 


-7 


-8 


KPSHARE 


b. 


Takes turns and shares? 


1 


2 


3 


4 


5 


-7 


-8 


KPCURIOS 


c. 


Is enthusiastic and curious in approaching 




















new activities? 




2 


3 


4 


5 


-7 


-8 


KPPENCIL 


d. 


Is able to use pencils and paint brushes? 


1 


2 


3 


4 


5 


-7 


-8 


KPSTILL 


e. 


Sits still and pays attention? 




2 


3 


4 


5 


-7 


-8 


KPALPHA 


f. 


Knows the letters of the alphabet? 


1 


2 


3 


4 


5 


-7 


-8 


KPVERBAL 


g 


Communicates his or her needs, wants, 




















and thoughts verbally? 




2 


3 


4 


5 


-7 


-8 



RECONCILIATION. 



If discrepancies need to be reconciled, ask 
RECONCILIATION. Else, go to CLOSE3. 



We ask you [display full question]. One time we recorded [display answer that comes 
first in original response categories] and one time we recorded [display answer that 
comes second in original response categories]. What is the best answer? 

[Display all original response categories.] 



CLOSE3. Those are all the questions I have. Thank you for your time. 
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